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Convex Cardinal Shape Composition 

Alireza Aghasi and Justin Romberg* 


Abstract 

We propose a new shape-based modeling technique for applications in imaging problems. 

Given a collection of shape priors (a shape dictionary), we define our problem as choosing the 
right dictionary elements and geometrically composing them through basic set operations to 
characterize desired regions in an image. This is a combinatorial problem solving which requires 
an exhaustive search among a large number of possibilities. We propose a convex relaxation 
to the problem to make it computationally tractable. We take some major steps towards the 
analysis of the proposed convex program and characterizing its minimizers. Applications vary 
from shape-based characterization, object tracking, optical character recognition, and shape 
recovery in occlusion, to other disciplines such as the geometric packing problem. 

Keywords: Image Segmentation, Chan-Vese Model, Mumford-Shah Functional, Shape-Based 
Modeling, Compressive Sensing, Sparse Recovery, Optical Character Recognition, Geometric Pack¬ 
ing 


1 Introduction 


In many machine vision applications, the main objective is to identify and characterize regions of 
interest in an image. For instance, in image segmentation, the main goal is to label and aggregate 
pixels that are visually or statistically related. Specifically, in the binary case |13p7], a given image 
D is partitioned into disjoint regions E and D \ E, so that “similar” pixels lie on the same segments. 
In object tracking, objects of interest are identified and traced in consecutive frames of a video 
stream. Optical character recognition (OCR) is another example, where the characters inside an 
image need to be identified and correctly labeled to model a natural perception. 

It often happens that the geometry of interest is composed of simpler building-blocks. Moreover, 
if we have a fixed collection of simple prototype shapes, being composed of few prototypes may be 
a way of regularizing the problem. 

Some interesting imaging problems can be viewed as characterization problems, where the target 
geometry can be approximated as a composition of known prototype shapes. For instance, in OCR, 
a word may be considered as the union of distinct or overlapping characters. Characterizing the 
inclusion in the image using alphabetic characters as the prototype shapes may allow us to identify 
the constituting letters. Another example is tracking multiple known objects across video frames, 
where they may overlap or become occluded by simple optical obstacles. In both examples, applying 
basic set operations among a number of prototype shapes allows us to express the target geometry. 

In the past few decades considerable effort has been devoted to variational models and level 
set type methods linked to the image segmentation problem [12,13,25 . Specifically in the context 


of using shape priors to perform the segmentation, the earliest work is reported by Leventon et 
al. 122 . The main idea is to setup the problem as an active contour evolution using a shape model 
prior. The parametric shape model is built by applying principal component analysis (PCA) on the 
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Figure 1: (a) An image segmentation problem with a dictionary of characters as the shape priors; along 
with the segmentation, the constituting characters are also identified; (b) Solving a puzzle by applying the 
proposed convex shape composition algorithm 


signed distance maps of the training shapes. The work was later improved by performing a direct 
optimization within the subspace spanned by the principal components [29,30 . Other frameworks 
such as direct application of the shape constraints to the zero level of the embedding function [ 13 ] , 
and simultaneously imposing shape priors to multiple objects in the image [l6] were considered to 
improve the segmentation efficiency. However, despite their widespread use, the majority of the 
level set-based techniques are cast as non-convex optimizations, where global solutions are often 
inaccessible. While the use of convex models for the image segmentation has been considered in 
some of the more recent works (e.g., see |7,10,ll|), incorporating prior shape information into a 
convex formulation has remained a challenge [28]. This paper will present a convex method to 
incorporate the geometry of prior shapes into the segmentation framework. 

In the context of binary image segmentation, consider a fixed collection of prototype shapes (a 
shape dictionary), and a given composition rule (such as the union operation). The segmentation 
problem of interest is determining a partitioner E that is representable by the composition of some 
elements in the dictionary. To promote geometrically simpler outcomes, the number of constituting 
elements in E may be limited to a fixed level. 

Despite its intuitive nature, the aforementioned problem is combinatorial, and its solution re¬ 
quires an exhaustive search among a large number of possibilities. Focusing on a composition 
rule that allows both component inclusion and exclusion (through the set union and set difference 
operations), in this paper we will study computationally tractable techniques that follow similar 
performance patterns as the combinatorial problem. Our main strategy is to provide a convex 
relaxation to the original problem, -eps-converted-to.pdf 

Before presenting the technical details of the segmentation model and the proposed convex relax¬ 
ation, we would like to motivate the reader by presenting some related results. Figure [lja) presents 
a challenging OCR problem where aside from the clutter present in the image, the characters are 
rotated and overlapped. Using lower and upper-case letters distributed throughout the imaging 
domain as the prototype shapes, we have shown the outcome of the segmentation when the number 
of constituting elements are limited to 4 (the top panel) and 6 (the bottom panel). The stem plots 
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next to each figure report the active elements in the segmentation, from which the underlying letters 
are identified. Among the most favorable features of our convex model are the simple formulation 
and the convenience in tuning the free parameter related to the number of active shapes (which 
often sweeps integer values). 

Figure [ljb) illustrates another binary segmentation problem, where E is simply a rectangular 
puzzle pad. The prototype shapes used in the dictionary are rotated and displaced elements of 
a puzzle. When the number of constituting elements is limited to 12, the optimal partitioner 
corresponds to the puzzle solution, which matches the outcome of our algorithm. Details for both 
examples are available in Section |5j 

After this quick overview, we will spend the remainder of this section to a more technical presen¬ 
tation of the problem, specifically focusing on the binary image segmentation task as a variational 
problem [f~2] . 


1.1 Shape Composition and Binary Image Segmentation 

For an imaging domain D c with pixel values u(x ), x e D , a binary segmentation corresponds to 
partitioning D into two disjoint regions E and D \ E, where each region encompasses similar pixels. 
A well-known variational model is determining E (and accordingly D \ E) via the minimization 

E* = argmin 7 (E) + / IR n (x)dx + / U ex (x) dx, (1) 

E JYj JD \e 

where 7 (E) is a regularization term promoting a desired structure, and II^ n (.) > 0 and U ex (.) > 0 
are some image-dependent inhomogeneity measures. 

A widely-used measure is the one proposed by Chan and Vese 1131, which takes IR n (x) = 
( u(x ) - Uin ) 2 and U ex (x) = ( u(x ) - u ex ) 2 , for scalar values Ui n and u ex . These scalars are chosen 
to be representatives of the average pixel value within each region. The resulting partitions E* and 
D \ E* are expected to aggregate pixels that take values around and u ex , respectively. The 
Chan-Vese model is in fact a particular case of the Mumford-Shah functional, used as a criterion of 
optimality for segmenting an image |24|. 

In a more general but closely related framework |[l5], the inhomogeneity measures are modeled 
as the log-likelihood function 

II r (x) = - logp(v(x)\ Or) r:in,ex , 

where v(x) is the similarity feature of interest (such as intensity, texture, etc) and 0 r parameterizes 
the probability density function p(.). Regardless of the approach taken, throughout the paper we 
keep the notation general and only assume 


\/x g D : n r (x) >0, r:in,ex. 

When U in and TI ex are fixed and known a priori, 

/ n ex (x) dx — / n ea r;(x) dx _ / n c *(x) dx, 

J _D\E JD J E 

and since the first term on the right-hand side is constant with respect to E, an equivalent formu¬ 
lation of ([!]) is 

E* = argmin 7 (E) + f (n in (x) - II ex (x)) dx. (2) 

e 
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In the majority of the techniques developed to address the segmentation problem 0 , the algo¬ 
rithms alternate between minimizing the cost and updating the inhomogeneity measures II^ n and 
H ex p~3{[T5] . We will take Ui n and U ex to be fixed and known, as our focus will be on solving 
the variational problem 0 . More details about the choice of the inhomogeneity measures will be 
provided in the simulations section. 

When the regularization term is included in the cost, additional structure inherited by the nature 
of 7 affects E*. Well-known regularizations used in the context of shape recovery often promote 
boundary smoothness and compactness on the resulting partitioner [I3p6 . 

As mentioned earlier, we consider a different way of regularizing the segmentation problem, 
where E inherits its geometric features from a set of prototype shapes. We consider E to be the 
result of a set-algebraic “composition rule” on a number of prototype shapes. 

Given a dictionary of shapes 2) = {Si, £ 2 , •••, S Us }, a shape composition rule, is a set algebraic 
expression (formed by basic operations, such as the union, intersection and set difference), which 
imposes a certain model for the objects sought. A natural choice for also used in (3], is 

We = ( USj)\( UM ( 3 ) 

jeZ© jeZ© 


where a combination of suitable elements in the dictionary through the union and relative com¬ 
plement reconstructs E. We require Z© and Z© not to be redundant, which intuitively means that 
removing any index from either set will change the resulting geometry (in Section 3.1 we will provide 
a precise definition of non-redundancy). 

With the dictionary in place, determining the non-redundant index sets Z© and Z e is the main 
objective of our segmentation problem, i.e., 


(SC) = argmin / (Tl in (x)-TY ex (x)) &x. (4) 

Z© Ze ^ Ax©, X© 

For n s elements in the dictionary, the number of possible regions that ^ can produce counts tc[^] 

, /c®, /c© 

Addressing 0 would therefore require a search among an exponentially large number of possibilities. 
We will henceforth refer to problem 0 as the basic shape composition (SC) problem. 

To promote simpler compositions and avoid shape redundancy, we will limit the number of 
shapes in the reconstruction. We regularize the SC problem by restricting the representation to at 
most s shapes 


£ 


k(n s - fc© - k e ) 


£ ( n d = 


= 3" s - 2” s + 1. 


(Cardinal-SC) min / (Uin(x) n-ecc(z)^ dz S.t. • |Z©| + |Z©| < S. (5) 

Z©,Z© J^x @ ,x e 

Solving ([5]) is the central focus of this paper and will be referred to as the cardinal shape composition 
(Cardinal-SC) problem. Addressing the Cardinal-SC problem also requires an exhaustive search 
among a large number of possibilities, specifically 


I+ | 1 O i - 1) - 

1 To count the possibilities, we consider the total number of ways to deposit the shapes into 3 bins and exclude 
from that the cases that the bin associated with Z© is empty 
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case^] This does not give rise to a computationally tractable process, especially for dense and 
over-complete dictionaries. We thus need to explore close alternatives to the Cardinal-SC problem 
that can be addressed in a computationally efficient manner. 

As we discuss in the next section, basic set operations among the shapes can be modeled by 
superimposing the corresponding characteristic functions. We exploit this property along with ideas 
from convex analysis to derive a convex proxy to the Cardinal-SC problem as 

min max (( n m(^) fl-ecc (x)(x) i (Ilin(s) ^ dx S.t. ||oi||i < T. (6) 

Here L^x) is a linear combination of the characteristic functions associated with the dictionary 
elements; 

n s f 1 T G S 

A*0) = 'ZoL jX s j (x), where = | • 00 

Roughly speaking, in relating the minimizer of © to the optimal index sets associated with the 
Cardinal-SC problem, the active ay values identify the active shapes in the composition and their 
sign determines the index set (X® or X©) they belong to. Inspired by ideas from sparse recovery, 
the i\ constraint is used to control the number of active shapes in the final representation. 

The main objective of this paper is to provide a more detailed presentation of © as a proxy to 
the Cardinal-SC problem. Our presentation is equipped with some level of analysis over basic setups, 
where the relaxation performs a similar job as the Cardinal-SC problem. Moreover, we will present 
generic sufficient conditions under which the proposed proxy perfectly recovers a target composition. 
Some of the techniques employed to analyze the problem are adopted from the compressive sensing 
and structured recovery literature |18|. However, the overall analysis theme, to the best of our 
knowledge, is the first of its kind and can be used as a benchmark for future extensions of the 
problem. The examples in the simulation section serve as the complementary justification to our 
analysis. 

1.2 Notation and Organization 

Our presentation mainly relies on multidimensional calculus. We use bold characters to denote 
vectors and matrices. Considering a matrix A and the index sets Ti, and T 2 , we use Ar 1?: to denote 
the matrix obtained by restricting the rows of A to Ti. Similarly, A : ^ 2 denotes the restriction of A 
to the columns specified by T 2 , and Ar 1 ,r 2 is the submatrix with the rows and columns restricted 
to Ti and T 2 , respectively. For a vector a - [ai, •••, a n ] T , we use oq to denote the i-th element of a, 
i.e., ai - ai. A quick reference table containing the notations mainly used throughout the paper is 
included in the Appendix. 

The remainder of this paper is organized as follows. In Section [2] we present a procedure that 
leads us to the convex relaxation of the Cardinal-SC problem. Section [3] is devoted to some basic 
definitions and preliminary results on the performance of the proposed convex surrogate. More 
extensive technical details and convex analysis tools are presented in Section [4] The main theme in 
this section is the possibility of recovering a target composition through the proposed convex proxy. 
Section [5] presents some experiments and simulations to support the ideas developed in the paper. 

2 To count the possibilities, we select k shapes from the n s shapes and count the number of ways to deposit them 
into two bins, excluding the empty X© case. The total number of possibilities requires a sum over k = 1 (plus 
the no-shape selection case) 
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(a) (b) (c) 

Figure 2: Representation of shape compositions via basic arithmetic operations on the corresponding 
characteristic functions: (a) union of two shapes, (b) difference of two shapes, (c) intersection of two shapes 


Section [6] ultimately provides the concluding remarks and discusses the possible future directions. 
To help with the flow, the majority of the technical proofs are moved to Section [7| 

2 A Convex Relaxation to the Shape Composition Problem 

This section discusses the rationale for using © as a convex proxy to the Cardinal-SC problem. 

2.1 Shape Composition by Superimposing Characteristic Functions 

In |[IJj3], Aghasi et al. introduced the notion of a pseudo-logical property. This property states that 
basic arithmetic operations among compactly-supported Lipschitz functions (called knolls in [3]) 
can approximately model certain set operations among the function supports. Due to particular 
differentiability requirements, the pseudo-logical property in | ljj3] was only discussed in the context 
of Lipschitz functions and remained an approximation. In this paper, we show that composing 
shapes can be exactly characterized by superimposing characteristic functions. 


Consider two given shapes S\ and S 2 and the corresponding characteristic functions xs-i and Xs 2 
as defined in ([ 7 ]). Basic set operations (u, \ and n) on Si and £2 can be related to basic arithmetic 
operation on the corresponding characteristic functions via the following three facts (illustrated in 


Figure [2|: 

supp + (aix 5l + CX2XS2) = <Si u S 2 

(aq > 0,a 2 > 0), 

(8) 


supp + (aixs! - a 2 Xs 2 ) = <Si \ <S 2 

(a 2 > aq > 0), 

(9) 

and 

supp + (aiXs 1 a 2 Xs 2 ) = <Si n<S 2 

(aqaq > 0). 

(10) 


Here supp + (.) denotes the positive support of the corresponding functions, i.e., for a given function 

X(x) 

supp + (x) = supp(x + ), (11) 

where y + returns the x values when y > 0 and zero otherwise. 


6 


Based on the preceding facts, a linear combination of characteristic functions 


L a (x) = Y, a iXs 3 {x), 

3 = 1 


could be a representative of %z@,T e for suitably tuned ay values. This is codified with the following 
result which is proved in Section [7| 

Proposition 2.1 Given a collection of shapes in D, for any composition E = ( U «Sj)\( (J Sj), 


there exist scalars ay such that L a (x) = Y a jXSj(%) satisfies 

3 €Tq U Xq 

L a (x) >1 x € E 
L a {x) <0 x i E 




j^e 


( 12 ) 


A corollary of Proposition 
between the values of L a 


2.1 is the existence of L a (x) such that supp + (L a (x)) = E. The separation 
~xj inside and outside E makes the representation stable. 


Using a function to characterize a region in a domain is a well-established technique in the 
imaging community, mainly discussed in the context of the level set method 1261. Considering a 
level set function 0(x), which characterizes the boundaries of a region E by its zero-level set (and 
takes positive values inside E), the image-dependent objective in (J2| may be written as 


f D (n in (x) - H ex (x))H(<p(x)) dr, 


(13) 


where H((jf) = l(o,oo) (</>), is the Heaviside function. Analogously, the smoothing and compactness 
regularizes can be cast in terms of <p to produce an overall functional merely in terms of <p (see 
examples in [ 5[[l3]). This way of modeling allows casting the segmentation problem as a variational 
problem in terms of 0. 

Regardless of the variational approach taken to carry out the minimization, based on Proposition 


2.1 and the functional representation (13), we suggest the following surrogate to the basic SC 
problem: 


n n s 

min / (n 

in (x)-H ex (x))H(L a (x)) dx s.t. L a (x) = Y i a j xs j (x), (14) 

“ JD 3 = 1 

where the ay coefficients are used to identify the shape composition elements. 

Regarding this model, few remarks are in order. First, the proposed surrogate requires searching 
for the minimizer in a subspace spanned by the shape characteristics, while the basic SC problem re¬ 
quires searching among a certain class of objects that follow the composition pattern <§• Based on 
Proposition |2.1[ we are however certain that all possible shape compositions have a representation 
in the subspace associated with the surrogate model. In other words, the optimization domain asso¬ 
ciated with the surrogate model entirely covers the corresponding optimization domain associated 
with the basic SC problem. 

Second, to characterize a region, our modeling relies on the positive support of the function 
L a (x)^ in spite of the level set method which concerns a function’s zero-level set to identify the 
region boundaries. The distinction is made more clear by noting that the zero-level set of L a (x) 
does not necessarily identify a region boundaries and can easily be a set with nonzero Lebesgue 
measure (e.g., see the green regions in Figure [2]). 
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We would like to note that the ultimate goal in our problem is identifying suitable elements of 
a shape dictionary, composing which according to Q yield an optimal segmentation. The level set 
discussions are mainly brought here to provide the proper transition between our framework and 
the classic segmentation techniques. 


2.2 The Convex Formulation 

The objective function in © is non-convex. In [TT] this problem is relaxed into a convex form by 
replacing H (</>) with a bounded version of 0, 

min / (n in (x) -U ex (x))</>(x) dx. (15) 

Jd 


With this setup, the optimal 0 function takes a value of 1 where IU n (x) < U ex (x) and vanishes 
where n^ n (x) > U ex (x). For any v e (0,1), the x-level set of the optimal </>(.) function identifies the 
optimal shape [Tl]. 

To bias the segmentation outcomes with the composition rule ^z 0 ,x e , similar to the previous 
section, one may think of replacing with L a (x) in (15) and cast the optimization in terms of a. 
However, identifying a shape through the x-level set of L a (x) for any v e (0,1), and still exploring 
the composition features (|8| and ([9| require Tq,(x) to take values beyond [0,1] (see equation ©)• 
In other words, we need to determine a reasonably tight convex formulation that pushes L a (x) 
towards [1, oo) anywhere that n i n (x) < H ex (x) and towards (-oo,0] where n i n (x) > U ex (x). 

To proceed with deriving a convex formulation, we rewrite the objective in (fTTl) as 


J d (jl in {x) - n ex (x))H(L a (x)) dx = J d (n in (x) - n ex (x)) + H(L a (x)) dx 

- J (n ex (a:) - n in {x)YH{L a {x)) dx. (16) 

For a fixed x, only one of the integrands on the right hand side of (Jl6| can be nonzero. Based 
on the sign of (n^ n - U ex ) we can consider a certain convex relaxation to each integrand. These 
relaxations are (U in -H ex ) max(T tt ,0) and (U in -H ex ) min(Tc, 1) when ( U in -U ex ) is respectively 
positive and negative (Figure |3|. An alternative relaxation to (Jm| is therefore 

(II in (x) - H ex (x)Y max (X a (x), 0) dx - (U ex (x) - II in (x)) + min (L a (x), l) dx. 


This objective can be written in a more concise form as 


X max (( n m(^) n! ecc (x))T/ q, (x), (n in (x) U ex (x)) ^ dx, 


(17) 


where n returns the n values when n < 0 and zero otherwise. 

Minimization of © is our proposed convex proxy to the SC problem. To impose the cardinality 
constraint on the number of active shapes, inspired by the ongoing trend in compressive sensing 
[8lj9][T8] , the cardinality of a may be controlled by imposing an ^-constraint on a. Ultimately, we 
suggest the following minimization as a convex proxy to the Cardinal-SC problem, and refer to it 
as the sparse-convex shape composition (Sparse-CSC) problem: 

(Sparse-Csc) min / max ( (n^ n (x) - U ex (x))L a (x), (n in (x) - U ex (x)) ) dx. (18) 

||o:|| 1<T JD V ' 





Figure 3: The proposed convexification based on the sign of Ib n - U ex 


The Sparse-CSC problem may also appear in a regularized form as 

min max (( n m(^) n ecc (x)( x), (x) ii ecc (x)^ ^ dx + a|| 


ah. 


(19) 


The parameters r and A are free parameters that control the level of sparsity. In this paper we 
specifically focus on the constrained form (18), as tuning r is often performed in a more controllable 
way. 

From an implementation standpoint, either one of the convex programs (18) or (19) are what we 
need to address, in order to perform a segmentation based on the sparse composition of prototype 
shapes. 


3 Preliminary Notions and Results 

We start this section by providing a more formal definition of the fundamental concepts such as 
shape, disjointness and the non-redundancy of a composition. In Section |3.2| we demonstrate the 
equivalent performance of the Cardinal-SC and the Sparse-CSC in the case of disjoint dictionary 
elements. As a second extreme case, in Section [T3] we discuss the similar performance of the two 
problems when we make an extreme assumption about the inhomogeneity measures. These results 
bring a more intuitive sense about the problem without getting involved in the complexity of the 
model. 

3.1 Basic Concepts 

For a detailed study of the shape composition problem, we need to provide solid statements about 
the fundamental concepts. We take the initial step by presenting our definition of a shape. 

Definition 3.1 Given the imaging domain D c R d , we call S c D a shape if 

(I) S is a closed set (hence, the union of finitely many closed sets) in D . 
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(II) S is not a null set, i.e., 



> 0 , 


where ji is the standard Lebesgue on M d . To express this property, throughout the 

paper we equivalently use the notation int(S) ± 0, where int{.) denotes the set interior. 


As a concrete example, in case of d = 2, a finite collection of closed discs can be viewed as a single 
shape even if they are disjoint. However, a line segment cannot be classified as a shape in M 2 , since 
it violates the second property. 

For two given sets Si and S 2 in we will often make use of the following notations: 

• Si ~ S 2 , when int(Si) = int(S 2 )> 

• Si $ S 2 , when int(Si) + in^S^)- 

These notations assist us to establish precise arguments regarding the shape composition problem. 
Pragmatically, they are the means to compare regions without taking into account the boundary 
points (and the null sets). We hereby introduce the notion of disjoint shapes which plays a major 
role in our subsequent discussions. 


Definition 3.2 Two shapes Si and S 2 are called disjoint if 


Si n S 2 ~ 0. 


In other words, the intersection of two disjoint shapes can at most be a null set. 

The non-redundancy of a composition is another concept that we alluded to earlier, but requires 
a more precise exposition. 


Definition 3.3 Given a set of shapes {Sj}j e z e uz e , a composition = (Ujez 0 £j)\(Ujez 0 £j) 

is called non-redundant, if excluding any shape from the composition results in a measure change, 


i.e., 


and 


V/eJ 0 : 


Vj'eX e : 



3.2 Disjoint Dictionary Elements 

Here we consider a simple scenario where we can easily establish that the solutions to the Sparse- 
CSC and the Cardinal-SC are identical. We consider the case where the dictionary elements are 
disjoint (viz., every two shapes in the dictionary are disjoint). The analysis of this problem will 
provide insight into the more general case of overlapping shapes. 

In the case of disjoint dictionary elements (DDE), the non-redundancy of %z@,z e requires X e = 0 
(see Definition 3.3). Consequently, the composition rule is simplified to = U jez @ $j and the 
Cardinal-SC problem becomes 


Tl - argmin / (U in (x)-U ex (x)) dx s.t. : |2®| < s. 


(20) 


3 In this paper we use the simpler notation f s dx to denote a volumetric integration. 
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Under the DDE conditions, Z^ can be uniquely determined by making mild assumptions about the 
average inhomogeneity measures over the shapes. For this purpose, given a dictionary of shapes 
{SjYj=i we define the following quantities for every dictionary element: 


Pj = f (U m (x)-U ex (x)) + dx>0, Qj= f (ll ex (x)-U in (x)) + dx>0. 
J Sn J Sn 


Clearly, when the dictionary elements are disjoint, for any 2® c {!,•••, n s } 



Based on this observation, we sort the shapes upon their (Pj - Qj ) values. That is, considering an 
index order ji,j2,'",jn s such that 


Pjl Qjl - Pjz Qj2 - ' - Pjm Q. 


< 0 < Pj m+1 Qj m+ 1 


- ’ - Pj'n* Q 


( 21 ) 


Determining Z ^ is now straightforward: 

Proposition 3.4 Given a set of disjoint shapes {Sj}™*i and the image dependent measures II i n (x) 
and U ex (x), consider the general progression (2l\ ) for the shape indices 1, •••, n s . If the s-th inequality 
in (2l\ ) is strict and s <m, the solution to the Cardinal-SC problem (2(fy is unique and 

P® = {ji->32, **■? js] • (22) 


We will skip the proof for Proposition |3.4| a s it is trivial. We, however, note that the condition 
s < m makes the sparsity constraint in (| 20 |) active. For s > m the Cardinal-SC problem (20) is 
equivalent to the corresponding SC problem, where the cardinality constraint is disregarded. 


Theorem 3.5 Suppose the assumptions of Proposition \3J\ hold. Then, for r - s, the minimizer to 
the Sparse-CSC program (denoted as ol* ) is unique and Z ^ = {j : a* = 1}. 

Proof: We use the disjoint property among the shapes (and accordingly the non-overlapping prop¬ 
erty among their characteristic functions) to simplify the underlying convex cost as follows: 


n n s 

/ (n in (x) - n ex (x)) max ( y oijXSj (x), o) da; 
JD i=i 

= f (n* n (a;) - II ea: (a;)) + ( y xSj ( x ) max(aj, 0)) da; 

JD V i=! 

r / n s 

= J D ( y {n in (x) - U ex (x)) + XSj (x) max(®, 0 )) da; 
= ymax(aj,0) J^(U in (x) -H ex (x)) XSj(x) dx 


= y p j max(o!j, 0). 

3 = 1 
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(a) (b) 

Figure 4: The plot of F(a) = Pmax(a,0) - Qmin(a, 1), which takes its minimum at 1 when P < Q and 
takes 0 as the minimizer when P > Q. 


In a similar fashion 


n + n s n s 

/ (n ea: (a;) - n in (x)) min ( ajXSj (x), l) dx = Q :j min(a j , 1), 
JD j=i i=i 


and therefore in the DDE < 


f max((n in (x) II e:c (x)(x), (ilin(^) n ecc (x)^ ^dx— Ey () 

7=1 


where 

Fj(a ) = Pj max(<a, 0) - Qj min(<a, 1). 

In other words, under the DDE condition the convex objective reduces to the sum of separable 
costs in terms of a components. As depicted in Figure [4j the underlying costs take their minima 
at 0 or 1, depending on the sign of Pj - Qj. Moreover, 


min Fj{ot) 

a 


Pj Qj 0 < Pj < Qj 

0 0 < Qj < Pj 


To minimize YJj=\ Fj ( a j ) subject to ||a||i < 5, for s < m, we need to assign unit values to the 
coefficients ctj 1 , oij 2 , •••, oij s and set the others to zero, otherwise, we are not minimizing the objective 
sum to the maximum extent. Comparing this result with (22) reveals that X® = {j : cx* = 1 }. | 


The preceding results confirm the possibility of using Sparse-CSC as a proxy to the Cardinal- 
SC problem for the proposed setup. Aside from the DDE condition, the mild conditions stated 
in Proposition |3.4| guarantee the uniqueness of the solution for the Cardinal-SC problem and the 
convex proxy, and allow us to show their equivalence. 
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3.3 Lucid Object 

In the previous section we considered an extreme case of non-overlapping dictionary elements. In 
this section we consider another extreme case, which reveals new facts about the similar performance 
of the SC problem and the convex formulation. 

Definition 3.6 For a given region He D, the lucid object condition (LOC) holds if 

f < U ex (x) X G E 

y ^ H-ex^p^) X € D \ E 

As an example, consider an image where the pixel values are bounded as 

j 0 < u(x) < | x e E 
| | < u(x) <1 x € D \ E ’ 

and the Chan-Vese inhomogeneity measures II i n (x) = ( u(x) - 1/4) 2 and U ex (x ) = ( u(x ) - 3/4) 2 are 
considered. It is straightforward to verify that in such setup, the LOC holds for E. 

When the LOC holds for E and there exists a composition of the dictionary elements linked to E, 
the outcomes of the SC problems and the proposed convex proxy maintain some general properties 
that will be detailed. 


Proposition 3.7 Consider E c D and a dictionary of shapes If the LOC holds for E and 

there exists a unique non-redundant representation such that E = then 


{Xf,Xf} = argmin / (il in (x) - Tl ex (x)) dx. 

X® ,X Q J ^ex©, x e 


An immediate interpretation of Proposition |3.7| is, under the conditions stated, the SC problem 
identifies the constituting dictionary elements associated with E. This result is readily generalizable 
to the case that there are multiple non-redundant representation %x®,x Q linked to E, and x s 
is the unique representation with the fewest number of elements. In this case 


{X® ,xf } = arg min / (n in (x) - U ex (x)) dx s.t. | X® | + |X e | < |X^| + |X| 

X ffi ,X e 


We now draw our attention to the Sparse-CSC and its performance under the LOC. 

Proposition 3.8 Consider EcD and a dictionary of shapes {SIf the LOC holds for E ; 
then for any ol e W ls the convex cost JTt[ ) is lower bounded as 

max (( n m(^) H-ex (^) (t) 5 (Urn (^) ^ dx > J^Tl ex (x') dx. 

As a matter of fact, the lower bound in Proposition |3.8| can be attained and the corresponding 
minimizer is related to E as follows: 



Theorem 3.9 Suppose the LOC holds for E and there exists at least one composition of the dic¬ 
tionary elements for which E = cI(%z®,Zq )• Then, a* minimizes the cost 

/ fl max (( n m(^) II e: £(*^))-L ck (x), (lb n (x) II ea ,(x)^ ^ dx (23) 
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if and only if 


A**( x )-1 x € int(Yi) 
L a * ( x ) <0 x e int(D \ E) 


( 24 ) 


From Theorem 3.9 we can immediately see that a * is not unique (e.g., if a* is a minimizer, ka* is 
also a minimizer for any k > 1). However, for any minimizer a* we have 

supp + (£. a »(a;)) = £, 

which means that by minimizing the convex cost © we can identify the underlying object E. 

The identification of E is the common outcome of the SC problem and the proposed convex 
proxy. However, the link that Proposition |2.1| establishes between an arbitrary composition and 
a vector satisfying (24), makes us hopeful about identifying the constituting elements of E by 


inspecting the entries of a*. In this context, an t\ restriction on a is capable of making a* unique 
and more conveniently linked to a specific composition. Moreover, such restriction allows us to 
control the number of shape elements appearing in the representation of E. 

The remainder of the paper is devoted to the analysis of the Sparse-CSC under more general 
conditions. The main theme is the possibility of using the proposed convex proxy as a tool to extract 
the constituting elements of a composition along with the segmented image. While a general 
framework is considered throughout the analysis, we will later revisit the LOC as a particular 
problem to which the proposed tools can be readily applied. 


4 Extended Tools and Convex Analysis 

Both the information embedded within the image, as well as the overlapping of the dictionary 
elements play key roles in characterizing the minimizers of the Sparse-CSC program. In this section 
we will have a deeper study of the convex problem, and provide some analysis tools to characterize 
the possible outcomes of the problem. 

In Section |4.1| we introduce a process that extracts the disjoint components of overlapping 
shapes, as a simplifying tool for the analysis of the Sparse-CSC problem. In Section p~2| we establish 
a bijective relationship between the representation of a composition in the shape domain and the 
a-domain. To construct a bijection, we relate the two representations via a linear program and 
discuss the uniqueness conditions for the proposed problem. In Section |4.3| we derive sufficient 
conditions under which a target vector a minimizes the Sparse-CSC problem. The derivation is 
performed in a general setup, where there are no restrictive assumptions about the overlapping of the 
dictionary elements and the spatial variability of the inhomogeneity measures. The developed tools 
are employed in Section |4.4| to derive sufficient conditions under which the Sparse-CSC program 
extracts the constituting elements of a target composition. 

4.1 Disjoint Shape Decomposition 

As detailed in Section |3.2[ for a dictionary with disjoint shapes it is quite straightforward to infer 
the solution to the Cardinal-SC problem and relate it to the solution of the corresponding convex 
proxy. To pave the analysis path for the case of overlapping shapes, in this section we propose a 
procedure to decompose overlapping shapes into non-overlapping ones and exploit that to approach 
the general shape composition problem. 
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Given n overlapping shapes <Si, £ 2 , ***? 5 n , consider an n- dimensional vector J e { 0 ,l} n \ { 0 } n . 
A set D with nonempty interior is called a shapelet if 

(25) 

3=1 


where J j denotes the j-th element of J, and 

©m(<5) = | d ( 5 c) m = 0 • (26) 

Here, cl(.) denotes the closure of the set. The appearance of cl(.) guarantees that the resulting 
shapelets maintain the properties of a shape. We will call J the constructor vector associated with 
D. As some of the intersections in the form of ( [25] ) might be null sets, the number of shapelets, 
denoted as %, is at most 2 n - 1 . 

We index the shapelets as and the corresponding constructor vectors as for i- 1,2, •••, uq. 
Furthermore, for every shape Sj we define the index set 

lj = {* : Jf = 1}, (27) 

which corresponds to the shapelets inside Sj. We henceforth use the terminology disjoint shape 
decomposition (DSD) for the process of generating the D shapelets from a collection of given shapes 
Si,S 2 ,-,S n . We succinctly write 


{n i ,J (<) }r-°i = DSD({5 i }? 1 ). 


(28) 


Furthermore, we may form a binary matrix B e {0,l} nriXn by stacking up the transposed 
constructor vectors from the DSD process (28); more precisely 


Bi,j = jf. 


When B is resulted from the DSD process over a specific set of shapes, we call it the corresponding 
bearing matrix , and when its construction involves all the elements of the dictionary, we refer to it 
as the dictionary bearing matrix. 


Proposition 4.1 Given n (overlapping) shapes Si, S 2 , ••*, S n , for the corresponding shapelets con¬ 
structed through the DSD process the following properties hold: 

(a) D i:L nf^ 2 ^ 0 , (for i u i 2 e {1,2,—,n n }, h * i 2 ) 

(b) U^i = U] =1 Sj, 

(c) Sj — U ieXj • 

Putting this into words, the DSD process performs a partitioning on each shape (ignoring the 
measure-zero boundaries). In the simplest case of two overlapping shapes Si and S 2 , the three 
possible shapelets are Di = cl(Sf) n <S 2 , D 2 = Si n £2 and D 3 = Si n 01(5^) for which one can easily 
verify the properties (a)-(c). Figure [ 5 ] provides another example for n = 3 where uq = 5 < 2 3 - 1. 
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i 

j(i) 

a = nU e j(;} (s 3 ) 

1 

[1,0,0] T 

Si n S 2 n <Sf 

2 

[0,1,0] T 

§1 n S 2 n 

3 

[1,0,1] T 

<Si n n S 3 

4 

[1,1,Of 

<Si n S 2 n Si 

5 

[1,1,1] T 

S\ n S 2 n S 3 


B = 


(1 

0 

1 

1 


\1 


0 0 \ 
1 0 
0 1 
1 0 
1 1 / 


Figure 5: An example of disjoint shape decomposition, where the shapes <Si, & and S 3 are decomposed 
into the shapelets The constructor vectors are listed in the table and the corresponding bearing 

matrix is presented as B. To highlight the pattern we succinctly used the notation O to represent cl(O). 
For this example X\ = {1,3,4, 5}, X 2 = {2,4,5} and X 3 = {3, 5} 


Theorem 4.2 For a given set of shapes {Sj }j € x 0 ux e , consider the shapelets resulted by applying 
the DSD process. If the composition %z®,x® - (UjeX e ^j)\(}JjeX e ^j) i s non-redundant, then 

(a) for each j e X®, there exists a (unique) shapelet fl c Sj such that 

\f j r e (X® u X©) and j' + j : fin Sj' ^ 0; 

(b) for each j e X®, there exist some shapelets fl c Sj n ( U s?zc/& that 

M j ' € X® and j' ± j : D n <Sy ^ 0 . 

As an illustrative example, Figure [ 6 ] shows the resulting shapelets of the DSD process on four 
shapes Si,---,<S 4 . The composition ^n, 2 },{ 3 , 4 } is a non-redundant representation. It can be readily 
observed that part (a) of Theorem |4.2| applies to fli c5i, and D 3 c <S 2 - Likewise, part (b) applies 
to D 5 c £ 3 , and to the multiple shapelets fl%, ^9 and ^11 which are in S 4 . 


4.2 A Bijective Relationship Between the Shape Composition and the 
Combination of Characteristics 


Proposition 2.1 establishes a relationship between %x@,x® an d To avoid an exhaustive 


search associated with the shape composition problem, our proposed strategy is determining the dj 
coefficients as a proxy to selecting the index sets X® and X®. For this purpose we need to provide 
a procedure that relates the two representations. More specifically: 


(1) Given a and knowing that L^x) corresponds to a non-redundant composition %x@,x®-> Low 
can we determine X® and X©? 
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(2) Given {Sj}™^ and the index sets X® and X®, how can we determine an a vector that is a 
representative for %z®,z e ^ 

To address the first question, for a given a we define 


V(«) - {j -j e {l,2,-,n s },aj > 0} 
£“(<*) - {.) '-3 e {1,2 ,-,n 3 },aj < 0} 


Knowing that L a (x) corresponds to an %z®,z Q , the natural choices for X® and X® are £ + (a) and 
£ _ (a), respectively. 

To address the second question, from Proposition |2 .1 1 we are always certain about the existence 
of an ol vector such that 

supp + (£ a (z)) = ^r e ,x e - (30) 

In fact, as inferred from the proof of Proposition |2.1[ there is always an infinite number of possible 
ol vectors that satisfy (30). As a requirement for our subsequent analysis, we will introduce a 
procedure that limits the corresponding a vectors to a certain set, potentially a singleton (a single 
element set). For this purpose, we will pose the process of determining the representative a as a 
linear program and will discuss the uniqueness conditions for the proposed problem. This process is 
closely linked to the proposed convex framework and specifically designed to facilitate the analysis. 

Given {Sj }j e x 0 ux e , we perform a DSD to generate a set of disjoint shapelets, i.e., 


{^V (i) }”il = DSD({5,} i€ x 0 ux e )- 

As the Cli shapelets are disjoint, and Sj = U ieXj f° r each j e X® uX®, we can write 

^ e ,x e = ( u S*)\( U S s ) = ( u U fii) = u a. 

jeZ® jeX e ie\JXj i€\JXj ie( \JZj)\( \JZj) 

i eX ® i eZ 0 J6X 0 jex© 


( 31 ) 


To simplify the future formulations, we will make use of the following notations: 

A® = U Xj and A© = [J Xj. 

j tZ@ j €Xq 

The sets A® and A© contain the indices of the shapelets that are in Ujez 0 Sj and Ujez e <Sj, respec¬ 
tively. Clearly, the two sets are not necessarily disjoint. 

In constructing an ol vector, we need to make sure that for every i e A® \ A© we have 


L a (x) >0, x efti 


and for every i e A©: 


L a (x) <0, x e fli. 


Conditions (32) and (33) basically assure that (30) is satisfied. We proceed by defining 


| = = \) 

{ Kf = 1 } ’ 


(32) 

(33) 


(34) 
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J (4) = [ 0 , 0 , 1 , of . 

(^3 < 0 

J (8) = [ 1 , 0 , 0 , if . 

a/i < -1 

J (9) = [ 1 , 1 , 0 , if . 

0^4 < -2 

J (11) = [ 0 , 1 , 0 , 1 ] T • 

04 < -1 


Figure 6: Finding the a vector corresponding to (Si u S 2 ) \ (S 3 u<S 4 ): based on the DSD applied, we 
have A/© = li ul 2 = { 1 , 2 ,3,5, 6 , 8 ,9, 11 } and A/© = I 3 UX 4 = (4,5, 6 ,7, 8 ,9, 10 , 11 }. After setting an and 02 
to 1 , we need to choose 03 and 04 in a way that in all shapelets marked with a black circle (indexed by 
A/©) the value of Lot(x) is non-positive. This requirement gives rise to eight linear inequalities in terms of 
0 L 3 and 04 . The linear constraints can be deduced by referring to the constructor vector of each shapelet. 
For instance, for flu we have /C© 11 ) = {2} and /C© 13 ^ = {4} and based on (36) the corresponding inequality 
is 0:4 < - 1 . 


The set (correspondingly K^q) contains the index of the shapes { Sj}jez 0 (correspondingly 
{ Sj}j€i e ) that overlap with 

If for j e X® we set otj - 1, then (32) is automatically satisfied. To assure that (33) holds, we 
need to impose 

Vie AT© : £ £ OLj< 0. (35) 


jelC. 


(i) 




Since a, = 1 for j e I©, (35) is equivalent to 


Vie W 


£ ©<-|4°l> 


(36) 




where | . | represents the set cardinality. The linear inequality constraints (36) define a convex set 
for the permissible otj coefficients when j e X®. These constraints are met when the aj values are 
sufficiently negative for j e X®. To limit the number of possibilities we consider solving the linear 
program 


max 

s.t. 


a* 


£ 

j^e 

£ 


a- 


<-\K 


(01 


VieVe ’ 


(37) 


which is capable of having a unique solution. 

Summarily, given {Sj }j e x 0 ux e , 2© and X®, determining the corresponding a is performed 
through the following linkage process : 


(I) for j e 2®, set ctj = 1; 
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/3 f = 


/ 1 0 0 \ 

0 1 0 

1 0 1 

1 
1 


1 
V 1 


0 
1 / 


/1 
1 


Figure 7: The function Lo,(x) as a representative of the non-redundant composition ^i, 2},{3}> for the 
shapes presented in Figure [5l Stepping through the linkage process for this composition simply yields 
^ = (1,1, -2 ) t and accordingly & = (1 1, -1,2,0) T 


(II) for j eZ 0; the aj values are selected to be a maximizer to the linear program (SI). 


We again make a reference to Figure [6] as an illustrative example. 

Corresponding to any vector at obtained through the linkage process, we consider a vector 
/?t € W as 

p\= y a ]> i = 


The quantities /?] are simply the value of C a t (x) over int(0,) (Figure [t]) . 
matrix associated with (31) as 


Denoting the bearing 


the shape and shapelet coefficients may simply be related via 


/3 f = 


(38) 


The following result states that when %x @1 x Q is non-redundant, regardless of the maximizer’s unique¬ 
ness in (37), there are some general properties that hold for of and (3*. 


Proposition 4.3 Given a set of shapes {Sj}j € z®ux e and the corresponding index sets Z® and Z® ; 
suppose the composition %x@,x Q = (Uj€X^j)\(UjeX e ^j) is non-redundant, af is a vector obtained 
through the aforementioned linkage process and (3 ^ = B^ol f is the corresponding shapelet coefficient 
vector. The following properties hold: 

(a) for each j e Z® ; at < -1; 

(b) for j gI 0; there exists a unique i e Zj such that /3t = 1 (unit-valued shapelet); 
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Si 




£5 


/ 

1 

0 

0 

0 

0 

f^2 


0 

1 

0 

0 

0 



0 

0 

1 

0 

0 



1 

1 

0 

0 

0 



0 

0 

0 

0 

1 



0 

0 

1 

0 

1 



0 

0 

0 

1 

0 



1 

0 

0 

1 

0 


V 

1 

1 

1 

1 

1 




(a) 


(b) 


Figure 8: (a) A combination of five rectangular shapes <Si,<S2, •••,<S5, carefully overlapped. The shapes £4 
and S 5 are each composed of two components connected by the dash-lines. For the composition 3},(4,5}, 
the linkage process does not produce a unique a-vector. Some possible outcomes of the linkage process are 
an = (1,1,1, -1, -2) t , 0 L 2 = (1,1,1,-2, -1) T and 0:3 = (1,1,1, -1.5, -1.5) T . The value of L ctl ( x ) over each 
shapelet is displayed in the figure, (b) The bearing matrix corresponding to the composition depicted in 
panel (a). Considering cti to be an outcome of linkage process, the resulting null-valued shapelets are Qs 
and fig. Although the discriminant matrix is full-rank, equation (42) does not return a strictly positive 
solution 


(c) for j zTq, there exist i eZj such that fit = 0 (null-valued shapelets); 


(d) the mapping (38) relating a t to (3' is injective 


‘Kzt 


When the maximizer of (37) is unique, the linkage process allows relating a given composition 
5 x e to a well-defined vector. In this case we may simply represent the relationship by 


uZ e ;2©,Z©). 


By construction, if = A({Sj} je z (B uZ e ',Z®,Ze) then £ + (a^) = 2© and £“(a^) = 2©. 

In general there is a possibility that stepping through the linkage process does not yield a well 
defined a^, due to the non-uniqueness issue. An example of this case is shown in Figure [ 8 ^a) to 
construct which we have performed a careful shape selection and alignment. 

A natural question would be: what class of non-redundant compositions supports the formation 


of A (or simply when (37) produces a unique solution)? In the following we will discuss that since 
the existence of the null-valued shapelets is an intrinsic property of any non-redundant composition 
(Proposition 4.3), under simple assumptions about the structure of the bearing matrix restricted 
to such shapelets, the uniqueness may be guaranteed. 

For the sake of convenience in the derivation, we consider a matrix representation of (37). It 
is straightforward to verify that for a non-redundant composition %x®,x Q i the bearing matrix is 
structured as 
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(39) 




B (2 ’ 1) 


0 

£^( 2 , 2 ) 


|A/©\A/"e 

}A/e 


where B £?( 2,1 ) anc [ £?( 2 ’ 2 ) are binary matrices. The linear program (37) may then be cast as 

(40) 


max 


l T a Xe 


s.t. 


}( 2 , 2 ) 


a Ie < --B 


( 2 , 1 )-, 


For an outcome of the linkage process, the unit-valued shapelets may be indicated by 


Likewise, if at is a maximizer of (40), we can indicate the resulting null-valued shapelets by 




>(2,2) t 


(24) n 


and correspondingly define a discriminant matrix as 


A %. - B 


( 2 , 2 ) 


(41) 


An argument of duality, presented in the proof of Theorem |4.5[ reveals that there always exists a 
vector w > 0 such that 

(42) 


— 1 . 


It will be discussed in the sequel that if the discriminant matrix associated with a non-redundant 
representation maintains certain properties, the linear program (40) attains a unique maximizer. 
We specifically focus on the non-redundant compositions that the number of null-valued shapelets 
does not exceed |X©|. 

Definition 4.4 A non-redundant composition is basic, if for a maximizer rank(A.%J 

Pel = l r o1 and the (essentially non-negative) solution of (U^|) is strictly positive. 


Theorem 4.5 If a non-redundant composition is basic, then the maximizer of (40) is unique. 
Conversely, if for a non-redundant composition |T^| = |X©| and the maximizer of (40) is unique, the 
composition is basic. 


Simply, applying the linkage process to a basic composition yields a unique and well-defined 
representation of the composition in the a-domain. As exemplified earlier, for the composition 
depicted in Figure [8^a), the linkage process fails to provide a unique representation. By analyz¬ 
ing the underlying discriminant matrix demonstrated in Figure [8^b), we easily observe that the 
composition fails to pass the required criteria of being basic. 

As a matter of fact, the uniqueness of the maximizer in ( |40| ) can be warranted for a broader 
class of compositions. Moreover, if of is an outcome of the linkage process for a given composition 
tKz®,Z qi by construction £ + (a^) = X© and = X©, regardless of the outcome’s uniqueness. 
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In other words, if an algorithm uses a as a proxy to identify the index sets X® and X®, once 
it recovers any outcomes of the linkage process, the identification is successful. Focusing on the 
basic compositions allows us to markedly simplify the analysis by exploiting some of their favorable 
properties. However, the main ideas developed in this paper are extendible to broader classes of 
compositions. 

For a basic composition, the conditions stated in Proposition |4.3| and Theorem |4.5| allow us to 
switch between different representations. Schematically, we can close the following chain 

A 

{^© 5 ^ 

e + ur 

where switching back and forth between different representations is conveniently possible. 

4.3 Sufficient Conditions for Unique Optimality of Sparse-CSC 

In analyzing the Sparse-CSC program a key problem is deriving sufficient conditions under which 
a target vector a minimizes the underlying convex objective. This section is devoted to addressing 
the aforementioned problem in a general setup, where there are no specific restrictions on the 
overlapping of the dictionary elements and the spatial variability of the inhomogeneity measures. 
We will show that a given vector a is a solution to the Sparse-CSC problem if a certain system of 
constrained equations can be solved, where the coefficient matrix is related to the dictionary matrix 
and the right-hand side depends on the image data. 

To maintain brevity we will use the notation 

G(a) = X max « I-Ccc(^))X q- (x), (n^ n (x) IIc cc (x)) ) dx, (43) 

and delineate the goal as deriving sufficient conditions under which a designated vector a* e M ns is 
the unique minimizer to the convex program 

min G(a) s.t. ||a||i<r. (44) 

CX. 

We are certainly interested in the case that a* is sparse. 

As appeared in Section |3.2[ the analysis of the problem becomes significantly easier when the 
dictionary elements are disjoint. This is certainly not the case in a general setup, however, through 
the DSD process we can always generate a super-dictionary of the shapelets where every two 
elements are disjoint: 

Accordingly, as illustrated in Figure [7J one may rephrase L^x) in terms of the disjoint shapelets 
as 

n s nn 

£cx(x) = Y,otjXs j (x) = YsPiXnAx), (45) 

j=1 i=1 

where 

n s ... 

Pi = £ jfoLj. 

J = 1 
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Consequently, the corresponding vectors can be linked through the dictionary bearing matrix 


(3 = Bex, 


( 46 ) 


where as discussed in Section 


4.1 


Bij - J 


« 


By plugging (45) into the convex cost (43) and taking a similar path as Section 3.2 


rewrite G(a) in terms of /3 as 


we can 


n<n 

G(a) = y IM max(/3j ; 0) - q t riling, 1) 


= £ 03 ) 


(47) 


where 


Pi = 


~ Jo, n ea ,(x)^ dx, (fo — (n ea ,(:r) n^ n (x)) dx. 


(48) 


In other words, at the expense of the additional linear constraints (46), the non-separable cost 
function G(a) can be rewritten as a separable objective G{(3). 

Aside from a separable representation of the convex objective, we suggest an additional sim¬ 
plifying step which privileges the possibility of modeling ( [44] ) as a convex program with linear 
constraints. The following result plays the major role towards this purpose. 


Proposition 4.6 Consider a target vector cx* e M n % supported on T (ex* ± 0 for j e T and cx^ c = 0). 
Let c e M ns be a vector such that Cj = sign(<a*) for j e T and ||crc||oo < 1. For r = ||a*||i ? cx* is the 
unique minimizer of the convex program 


min G(a) s.t. ||a||i<r, 

cx 


if the following conditions hold: 

(I) ex* is the unique minimizer of the convex program 

min G(a) s.t. c T cx = r, (49) 

cx 

(II) the convex set C = {a : G(a) < G(a^)} has a nonempty interior and 

3deT c (cx*) s.t. c T 8 > 0, 


where Tc(cx*) denotes the tangent cone of C at cx* 


Basically, Proposition |4.6| states that instead of inspecting the conditions under which C touches 
the G-ball at a*, one may look into the problem of C touching a separating hyperplane between 
the two sets, at a*. Condition (II) of the proposition simply assures that the two sets are standing 
on opposite sides of the proposed hyperplane (as illustrated in Figure [9]). 

Supposing that one could verify the tangent cone property (II) for a target vector a*, property 
(I) allows us to focus on the convex program (49) which enjoys a simple linear constraint. This 
constraint may be combined with (46) through 


B 

XT 


CX = 


p 


(50) 
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Figure 9: A schematic of the sufficient conditions indicated in Proposition 


4.6 


which compactly presents the governing linear constraints on the (a,/3) pair. 

When tiq » we will have a left kernel for the left-hand side matrix in ( |50| ). Accordingly, 
we can build a matrix N e ]^( n ^ +1 ) xn ^ 5 columns of which form a spanning set for Null([B T , c]). 


Applying N 1 to both sides of (50) annihilates a and leaves us with a linear set of constraints 


merely in terms of (3. More specifically, 


n t 

A 

= n t 

B 

T 


r 


C 


a = 0 . 


(51) 


Making use of (51) along with (47), allow us to address the convex program (49) in the /3-domain 
via 


nun G{(3) 


s.t. 


N 


= 0 . 


(52) 


Unlike (44), this convex program enjoys a linear set of constraints and a separable objective, which 


significantly simplify the analysis. In Theorem 4.7 we will derive a sufficient set of conditions under 


which the minimizer of (52) is unique. These conditions are directly expressed in terms of B and 
c, and the appearance of N in the formulation is mainly for the purpose of clarifying the rationale 
behind the variable change. Before presenting Theorem |4.7| we need to proceed with introducing 
some new notions. 

Consider a potential minimizer a* and correspondingly (3* = Boc*. We assume that all the /3* 
values lie outside the interval (0,1). This assumption is consistent with the arguments presented in 
Section 


4.2 


where the a vector assigned to a given composition maintains such property. 

Based on the value of each entry in (3*, we may partition the set {1,2into four disjoint 
index sets denoted and labeled as follows: 


J r 0 - ={i:/3i<0,je{l,2,-,n n }} 

| r 1+ ={*:&> l,ie {l,2,-,n n }} 


(the support set) 


(53) 
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and 


| r 0 = {i:ft = 0,ie{l,2,-,n n }} 
{ r 1 ±{*:A = l ) *6{l ) 2,-,n n }} 


(the off-support set). 


We also introduce the bounding vectors l and u e Ml r ° uri l with entries 


^m(i) ~ 


qe-pe 0 

-Pi i e Ti 


and 'U'm(i) 


Pi I £ r o 

qi~Vi I € I'd ’ 


(54) 


(55) 


where ra(.) : To uTi i-» {l,2,--*,|rouTi|} is a simple bijective map that allows filling in the bounding 
vector entries in an order. 

Finally, we define a so called LOC violation vector e e M n % evaluated through 

e j - E It, j = 1,2,-,n s . (56) 

£€X / -nr 1 + ^€Xj-nr 0 - 


The LOC violation vector in some sense weights the LOC disruption over each shape. Basically, 
when the LOC holds and a* is a vector satisfying (24), e = oQ 


Theorem 4.7 Given the dictionary bearing matrix B e {0, l} n ^ xns ; consider a target vector a* € 
M ns that is feasible for (49) and correspondingly (3* = Bol* , such that all entries of [3* lie outside 


the interval (0,1). Further, consider c e M ns constructed as stated in Proposition 4-9. If the matrix 


[(■Brouri,:) 1, , c] has full row rank and there exist 77 e Ml r o ur il an y 77 c € M that satisfy 


and^\ 



r 0 ur 1 



(57) 


l < T] < U, 


(58) 


then, a* is the unique minimizer of the convex program (49). 


Altogether, in order to show that a* is the unique minimizer of the ^-constrained problem (44), 


we need to show that the tangent cone property of Proposition [T6j and the uniqueness requirements 
of Theorem |4.7| can be established simultaneously. To make a more intuitive sense of the material 
presented, we conclude this section with a simple toy example. 

Example 4.1. Consider the three shapes <Si, £2 and S 3 depicted in Figure [To} The resulting 
shapelets are shown and denoted by fli, For simplicity, the inhomogeneity measures are 

somehow that p\ - 1 - q\ - 0, and qi = 1 - pi = 0 for i = 2,3, •••, 7. In other words, the LOC holds for 
^1 - cl (Si \ (£2 u <£ 3 )). 

Suppose we add a forth shape to the dictionary and carefully select it to be fix, i.e., S 4 = 
cl(f2i). Focusing on the ^i-constrained problem (44) for a fixed r - 3, we can easily observe that 


4 It is straightforward to see that e also vanishes when ct* is a minimizer in the case of disjoint dictionary elements 
(described in Section [3.2| ), basically because Ti+ - To— 0. 

5 The rows of B r 0 uiT,: also need to be arranged according to the index map m(.) used in (|55|). 
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B = 


0 0 
1 0 
0 1 
1 1 
1 0 
1 1 
0 1 


Figure 10: The shapes used in Example 4.1 and the corresponding bearing matrix. The values shown in 
light color are the entries of (3\ = Boc\ over each shapelet when = [1, -1, -1,0] T 


aq = [ 1 ,- 1 ,- 1 , 0 ] T and aq = [ 0 , 0 , 0 , 1 ] T are two possible minimizers, as they are both feasible and 
satisfy condition p4l) of Theorem |3.9| As will be detailed in the sequel, we show that there is no 


way to establish the conditions of Theorem 4.7 for aq 
According to the figure, for the given value of a 


[1,-1,-1,C4] t where |c 4 | < 1 and therefore equation (157^ 


, To = {2,3} 
appears as 


and Ti = { 1 }. Also, 


c = 


A 1 1 
0 1 0 
0 0 1 
1 0 0 


1 

-1 

-1 

c 4 


V 

Vc 


= 0 . 


On the other hand for the bounding vectors we have 


-Pi 


' 0 ' 


Y 

to 

1 

to 

= 

-1 

and u = 

0 

Q3 


-1 


0 


(59) 


(60) 


We can see that for any c 4 € [- 1 , 1 ], equation (59) has the unique trivial solution 77 = 0 , r\ c = 0, and 
given the bounding vectors specified in (60), there is no way to establish condition (58). 


4.4 Accurate Recovery of the Shape Elements 

The tools developed in Sections |4.2| and |4.3| can be employed to characterize the minimizers of 
Sparse-CSC, and explore the possibility of identifying the constituting elements of a target com¬ 
position. Since considerable attention has already been drawn to the derivation of the proposed 
convex proxy and the related analysis tools, we will consider a setup pertinent to the preceding 
example that allows integrating the techniques in a concise, yet insightful way. The ideas developed 
here may be further extended to derive more general results. 

Consider a dictionary of shape elements {Sj }™:q and an image D, where the LOC holds for ScD. 
We assume that there exists a basic non-redundant composition f% e ,z e such that X®,X e c { 1 , •••, n s } 
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and E = cl(f% 0? x 0 )- Concerning the basic composition, the outcome of the linkage process is 
represented by 

&X = W® uXe; 2®, 1q ). 

To avoid indexing complications, we simply assume that X® = 1, •••, n© and X® = n® + l, •••, n®+n®. We 
use the terminology of exterior for the off-target dictionary elements, index by {n® +n© + 1, -,n s }. 

We will assert that when n s = n© +n®, for r = ||w4({<Sj}j G x 0 uX e ;X©,X©)|| 1 , the unique outcome 
of the Sparse-CSC is a* = a^. Moreover, for n s > n© +n®, if the exterior shapes are placed in 
an “unstructured” way and maintain a restricted level of overlap with {<Sj}j € x 0 uz e > the Sparse-CSC 
program still attains a unique minimizer a*, such that c*x 0 uz 0 - and e*(x 0U x 0 ) c = 0- I n a sense, 
under such conditions, the exterior shapes cannot distract the convex proxy from identifying X© 
and X©. 

Consider and B ^ e {0, l} n ^ x ( n © +n ©) to be the shapelets and the bearing matrix 

associated with the DSD process 

{ftf = DSD (} j € x eU i e ) • 


When a shape is added to the collection {Sj}j € z® ux e , depending on its overlap with the present 
elements, the outcome of the DSD process may change to finer partitions. Strictly speaking, when 
n s > n© + n© and 

{fii,-Bi, : }2i=DSD({5 i }?' 1 ), 

there exist index sets such that 


nf = 


U a, 

ieJt 


£= 1 , 2 , 


; n n 


(61) 


To avoid confusion, we will refer to the super-shapelets as cells. 

The DSD process generates disjoint cells from the elements in the dictionary. For the sake 
of discussion in this section, it might be more convenient to look at a reversed process. Here a 
collection of disjoint closed sets, yet referred to as cells, is fixed and each shape in the dictionary is 
constructed by making a union over a number of them. 

Although the fixed cells are not generated by any specific DSD process, we will still denote them 


by (and note that (61) holds). This setup allows us to more conveniently track the process 

of adding a new element without updating the shapelet architecture every time. The only update 
made on the dictionary matrix would be adding a new column, while the number of rows always 
remains to be uq (see Figure [IT)). 

While we do not consider any specific geometry for the cells, at the finest partitioning level, the 
cells may simply be taken as the image pixels, and each element of the dictionary to be a collection 
of pixels. We assume each cell contributes in the construction of at least one shape (hence, the final 
dictionary matrix does not contain zero rows). 

As we will discuss, the null and unit-valued shapelets play a key role in recovering a basic 
composition through the Sparse-CSC. Denoting the unit and null-valued shapelets by and 
a key property of basic compositions is presented as follows. 

Proposition 4.8 Let c e M n © +n © ; where Cz @ = 1 and Cz e = -1. Considering the basic non- 
redundant composition f% 0 ,x e ; the linear system 


(b‘\ x ) T w = -c 

v rjurf,:' 


(62) 
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B = 


/ 10 1 
1 0 0 
1 0 0 
1 1 1 
1 1 0 
0 0 1 
0 1 0 
0 1 0 
0 1 0 
0 1 0 
V o o i 


\ 




Figure 11 : Left: the bearing matrix associated with the overlapped shapes <Si and S 2 . Right: a collection 
of fixed cells indicated by Hi, fin, which can be used as the building blocks for Si and £ 2 . The shaded 
cells form a third shape £3 = Uz=i, 4 , 6 ,n The overall dictionary matrix B is presented 


has a unique solution, which satisfies -(l + n©)l< w r ^ < -1 and 0 < w r ^ < 1. 

1 1 1 0 

As may be noticed from the proof of Proposition |4.8[ a basic composition i e has exactly n 0 
unit-valued (and n e null-valued) shapelets. Solving~~p2|) for w assigns negative quantities to the 
unit-valued (and positive quantities to the null-valued) shapelets. We will refer to the entries of w 
as the bearing constants. The purpose of considering the specific equation (62), and the connection 
between w and the shape identification problem will be revealed later in this section. 

Now, consider a vector a* e M ns such that 


a X 0 uX 0 - 


and 


OL 


(XquX©)' 


= 0. 


(63) 


Focusing on Proposition |4.6| and Theorem |4.7| we proceed to derive sufficient conditions for a* to 
be the unique minimizer of the convex program 


min G(a) 


s.t. 


<2 1 


- |^-({ < 5i}ieX 0 uX e ;^©,^'©) 


(64) 


where G(.) is our proposed convex objective (43). We begin by showing that when the LOC holds 
for E = cl(^j 05 x e ), the tangent cone property of Proposition 4.6 can be conveniently established. 


Lemma 4.9 Given a* as stated in (63), consider a vector c e W ls such that Cx 0 ux e = s ig n ( a x 0 ux e ) 
and ||c(x 0 ux 0 ) c ||oo < 1. There exists a point a inside the convex set C = {a : G(a) < G(a^)} such 
that c T (a - cx*) > 0. 

Since the lemma warrants the tangent cone property, based on Proposition |4.6| we would only 
need to focus on the uniqueness conditions for the minimizer of (49). 
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Suppose that (3* e M nQ contains the values of C OL *{x) over the cells Let T denote the 

index set associated with the cells that do not overlap with the constituting elements of E, i.e., 

T = {i:n i c((J«S;)\( U 

3=1 3=1 


By looking at the image of a* in the /3-domain, we expect /3* to match the values of C a ^(x) over 
the cells within U™?i" ne Sj and to vanish over the remaining cells. More specifically, 


ft = 


Pf i 6 Jt, 

0 


^ 1 0 ’ 1 1 ? 
ieT 


r 


% 

0 -’ 



Similar to (|54[), the off-support index sets associated with (3* may be indicated by 

and 


r 0 =Tu |J J t 


Ti = U Jt- 


ter? 


Using Theorem |4.7[ we will show that the shape identification condition is closely related to the 
structure of the dictionary matrix, and the overlap between the exterior shapes and the composition 
elements. To present the result in a more propitious and intuitive way, we proceed by introducing 
the related matrix blocks and an overlapping measure. 

Let us assume the cell index assignment is performed in a way that TouTi = {1, 2, *•-, |To uTi|}. 
Such assumption would avoid index mapping complications. Now, consider the binary matrix 
B' £ {O,l} (n0+ne)x l r ° uri l constructed as 


j,l IjOiCvSj} ^ Q S ■ ’ ^ ^J *’*) ^® ^0 5 ^ ^ r 0 U Tl . 

The columns of B' are clearly zero over T. One can verify that the remaining columns are multiple 
replications of the columns of ( B ^ ^ ) T , which yields 

rank(B') = rank(.B^ ur j .)=«©+ n e . 

We may follow a similar pattern for the exterior shapes to construct a matrix B" as 

B j-n S) -n e ,i = l {n^s j }, 3 = n B + n e + l,-,n s , ie TouTi. 

As a measure of overlap between the exterior shapes and the null and unit-valued shapelets, we 
define the quantities € [0,1] as 

lt,j = At y ieV^-urf-, j = n 9 + n e + l,-,n s . (65) 

\Jt\ izji 


These quantities are simply the relative number of cells that are common between the shapelet 
and an exterior shape Sj. 
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Theorem 4.10 Following the preceding setup, suppose E = cl{%^ r 0 ,x©) satisfies the LOC and 
^Te.Ie is a basic composition. If the cellular standing of the exterior dictionary elements is in 
a way that 


B' 

B" 


( 66 ) 


has full row rank and 


E 74 


w e 


< 1 


Vj e {n©+ne + l,-,n s }, 


(67) 


^rjurf- 


then the unique minimizer of the convex program (64) is ol* , which satisfies ol x uX 


a (I®uI e )<= - 0 


The magnitude of the quantity 


Cj = \ 


E j e (2® ul e ) c , 


olc^ and 


( 68 ) 


^rjurf 


is somehow related to the way an exterior shape overlaps with the elements of the target composition. 
The critical shapelets that contribute in the value of Cj are only the null and unit valued shapelets 
associated with ^j 05 x e - When the exterior shapes stay away from the elements of f% 0j x e , the result 
of Theor em |4.10| is quite consistent with what we expect to observe as the outcome of the convex 
program (|64|). We can somehow describe Cj as the “geometric coherence” between an exterior shape 
Sj and % e ,z e . 

An intuitive interpretation of Theorem 4.10 is upon having a dense grid of fixed cells (n^ ~ 
|To uTi| » n s ), when the exterior shapes are posed in an unstructured way (to satisfy the row-rank 
constraint) and maintain a sufficiently small level of overlap with the elements of f% 0 ,x 05 we can 
expect to identify the composition elements through the proposed convex formulation. As a matter 
of fact, it can be shown that the exterior shapes completely disjoint with \J™®(( ne Sj are even exempt 
from the row-rank condition of the matrix presented in (66). 

In view of w r ^ < 1 < |u; F ^|, overlapping of an exterior shape with the null-valued shapelets 

i o 1 - 


maintains a lower risk of violating (67) than overlapping with the unit-valued shapelets. Moreover, 


since the bearing constants take opposite signs over the null and unit-valued shapelets, an exterior 
shape that overlaps with both class of shapelets yet has the potential to maintain a small geometric 
coherence. While we have specifically focused on the case that the LOC holds for E, a line of 


argument similar to the proof of Theorem 4.10 would allow us to talk about the possibility of 
identifying the composition elements when the entries of the LOC violation vector (see equation 
(56)) are sufficiently small. 

An interesting problem arises when the exterior shapes are constructed through a random se¬ 
lection of the fixed cells. Bringing randomness into the problem would allow us to derive more 
qualitative results regarding the performance of the convex formulation in different setups, as well 
as relating the minimizers of convex proxy to the outcomes of the Cardinal-SC problem under 
certain regimes. While this type of analysis is very well-established in the compressive sensing 
community (9j[l8], we would leave that as a future work due to the present load of the paper. 

Finally, an essential piece of information that plays a key role in the Sparse-CSC formulation is 
the value of r. Based on the suggested formulation in (64), setting rtor* = \\A({Sj}jex^uX e ^®^e)\\i 
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grants the possibility of recovering the constituting elements of a composition. Generally, the value 
of r* is not known a priori, however, the fact that for many basic compositions the entries of 
are simply integer quantities, limits the selection of r* among integer possibilities. In the simplest 
case, when X e = 0, the value of r* is |Z®|, the number of elements in the composition. In the next 
section we will present some related simulations which demonstrate that trying successive integer 
values for r* may allow us to control the number of active elements in the recovered composition. 


5 Simulation Results 

In this section we test the efficiency of the proposed scheme in some basic examples. For the current 
simulations we have used the CVX Matlab toolbox 120], and have tried to maintain a moderate size 
for the shape dictionary Exploring more extensive simulations (emphasizing on the optimization 
tools to address the Sparse-CSC problem) remains a future work. 

The inhomogeneity measure used throughout the simulations is the basic Chan-Vese, where 
n in/ex ( x ) = ( u(x ) - u in j ex ) 2 . For images that are close to being binary, the values of £q n and u ex 
are fixed and simply set to the minimum and maximum intensity values in the image. For the 
more noisy images (later specified in the context), £q n and u ex are naively selected to be the 15% 
and 85% quantiles of the image histogram. In other words, the promising results presented are 
obtained with the least effort on optimizing the texture measures, and future developments in this 
area would further strengthen the technique. 


5.1 Basic Image Segmentation 

As the first experiment, we consider an image segmentation problem where very little information 
about the image content is available. In this case the elements of the shape dictionary are simply 
selected to be square blocks of the same size distributed throughout the imaging domain. 

A test image of size 120 x 100 pixels is shown in Figure p^a). Each element of the shape 
dictionary is a square of size 15 x 15 pixels, centered at a certain location in the domain. The 
shape centroids are taken to be on a uniform 40 x 30 grid over the imaging domain (i.e., 3 x 3| 
pixels spacing in the vertical and horizontal directions), which together result in a dictionary of size 
n s = 1200 shape elements. 

Since an object identification is not a primary goal of this problem and we are not seeking 
to match the content of the image with the dictionary elements, we simply use the regularized 
form of the Sparse-CSC model, parametrized by A, as specified in (19). The segmentation results 
are compared with the convex constrained Chan-Vese (CC-CV) model proposed in |111, where an 


optimal partitioner, characterized by 7r*(x), is obtained via the minimization 

= argmin / (Hi n (x) - U ex (x))7r(x) dx + A s / \\7tt(x)\ dx. 

0 < tt (^)<1 


(69) 


The smoothing penalty parameterized by A s controls the geometric complexity of the resulting 
partitioner. For both segmentation models the quantities £q n and u ex are set to be the 15% and 
85% quantiles of the image histogram. To provide a qualitative comparison, the segmentations are 
performed for low, mid-range and high values of A and A s . 

6 Examples of the code are available at: http://users.ece.gatech.edu/aaghasi3/software.html 
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Figure 12: Segmentation results on the test image in panel (a); left column corresponds to the CC- 
CV model; middle column corresponds to the Sparse-CSC model and the right column shows Lct(x) for 
each result in the middle column; (b,c) small penahg: A s = 1CT 3 , A = 1CT 2 ; (e,f) medium valued penalty: 
A s = 1CT 1 , A = 1; (h,i) large penalty: A s = 1, A = 10; 





































From a qualitative stand-point, a comparison between the outcomes of the CC-CV model and the 
Sparse-CSC problem (the left and middle columns in Figure [l2]) reveals that the latter is capable of 
producing simpler partitioners and is less sensitive to local details. For instance comparing Figures 


12'b,e) with [I2[c,f) shows that the Sparse-CSC model does not overreact to small image details 


such as the tree branches. In fact, the restriction of choosing elements from a dictionary provides 
a natural regularizer. Specifically, in the case of square blocks (super-pixels) used as the shape 
elements, one expects to observe a block-wise structure in the resulting partitioners. 

Another interesting observation is the outcomes of the two models when a simple geometry is 
sought (large values of A and A s are considered). Increasing the length penalty in ( [69] ) causes the 
resulting partitioner to bypass the local details at the expense of producing smooth corners (Figure 
HUM), while Sparse-CSC can yet track the details in a block-wise way (Figure [l2fc)). 

In general, increasing the i\ penalty in the Sparse-CSC problem controls the complexity of 
the resulting L^x) (the right column in Figure 12). A small penalty allows the finer details to 
be captured and a larger penalty provides a coarse segmentation highlighting the principal shape 
components. Using the CVX toolbox, each Sparse-CSC segmentation takes slightly more than 4 
minutes on a desktop computer with a 3.4 GHz Intel CPU and 16 GB memory. We would like to 
note that implementing a specific solver using the subgradient information (e.g. [I]) can significantly 
reduce the computation time, however, given the current load and theoretical focus of this work, 
such extension is left to a future work. The remaining set of experiments take advantage of the 
shape identification property of the Sparse-CSC problem. 


5.2 Image Segmentation with Missing Pixels 

As a more challenging example, we consider an image segmentation problem with a significant 
portion of the pixel values missing. Basically, the information about the pixels is only available 
over a subset of the domain, D' c D, while the segmentation needs to take place, globally. Prior 
information about the geometry of the objects in the image would allow us to yet perform a 
successful segmentation, as well as identifying the principal shape components inside the image. 

Figure [U^a) shows a noisy image before significantly discarding the pixel information, and solely 
presented as a reference image. The Gaussian noise causes a relative discrepancy of approximately 
24% compared to the ideal binary image. The pixel removal applied is in the form of rectangular 
patches and random samples. The resulting image is shown in Figure pB^b), which will be used in 
the first set of experiments. 

Figure [l3]c) shows a second noisy image, where the mean intensities differ for the objects 
present in the image. The Gaussian noise is higher in this case, causing a relative discrepancy of 
approximately 90% compared to the ideal piecewise-constant image. The pixel removal pattern is 
identical to the pattern in Figure [l3|fo). The resulting test image is presented in Figure [T3](d) , 
which will be used in a second set of experiments. 

Our prior information about the geometry of objects in the image is reflected in the selection 
of the dictionary elements. To build up the dictionary we make use of four basic shapes: a circle, 
square, triangle and an ellipse, as shown in Figure pB^e). The shape dictionary consists of n s = 999 
instances of these four shapes at various size and locations. In Figure p43^f), we have shown how a 
suitable composition of the objects in the dictionary would ideally reconstruct the objects in the 
reference image. It is however worth noting that, aside from the large missing pixel information, 
there is still a misalignment between the shapes in the dictionary and the objects appeared in the 
image. 
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Figure 13: (a) A noisy grayscale image to be used as a reference; (b) The image used for the first set 
of simulations, with the majority of the pixels missing as patches and random samples; (c) A more noisy 
image to be used as a reference; (d) The image used for the second set of simulations, with a similar pattern 
of missing pixels; (e) The shapes used to build up the dictionary: instances of these four shapes with 
different sizes are placed throughout the imaging domain; (f) The way the shapes in the dictionary need 
to be combined to form the objects in the reference image. Taking away the red shapes (with dashed-line 
boundaries) from the gray ones would provide us with the target objects 
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Figure [Mjpresents the segmentation results on the first test image following the preceding setup. 
To highlight the elegant performance of the method, we have shown the results for various selections 
of r in the optimization. The values of r are successively increased by integer steps until a full 
reconstruction is accomplished for r = 8 (Figure (l4^d)). It is striking to see how the objects are 
identified one after the other, as r increases and the convenience in tuning this parameter as an 
integer parameter. 

From a technical standpoint, based on the composition shown in Figure (l3^d), the value of 
II {$j }jgX 0 uX 0 ;X 0 ,2'©)||i is 8. Setting r to this value yields a set of objects that are comprised 
of very few geometric components and yet reasonably perform the segmentation task. While the 
optimal value of r is not expected to be known a priori, the fact that in many scenarios this value 
is simply an integer facilitates the algorithm with a convenient tuning process. 

Figure [l4](e) depicts the segmentation result when r steps beyond 8, yet indicating that the 
segmentation maintains a reasonable geometry and only few more shapes have become active to 
further reduce the convex cost. In Figure 14 T) we have employed the method presented in |3], 
highlighting that due to the lack of control over the sparsity level and the non-convex nature of 
the approach taken in (3], the results stand in a lower quality level as the ones acquired using the 
presented technique. Extracting the principal shape components from the outcomes of Figure [l4](f ) 
does not necessarily lead us to a reliable result. 

Figure [15] presents the segmentation results for the second test image, where the noise corruption 
is more severe and the mean intensities vary for different objects. For this case u ex and tq n are 
selected to be the 15% and 85% quantiles of the partially observed image histogram. Similar to the 
first set of experiments, incrementing r by unit steps causes the shapes to become identified one 
after the other. 

It is interesting to note that objects which contribute less to a decrease in the segmentation cost 
have the least priority in being identified (e.g., the doughnut-shaped object). Moreover, since the 
LOC violation is more significant in this experiment, for r = 8 we observe some level of difficulty in 
identifying the ellipse. More specifically, unlike the previous example in which the coefficient of the 
ellipse characteristic was recovered to be -2 (see the plot of a in Figure (l4^d)), in this example it 
remains to be -1 and instead a smaller ellipse inside it becomes negatively active. Basically, due to 
the low image contrast, fully taking out the large ellipse causes some of the pixels highly contributing 
to the cost reduction to stay outside the partitioner. For r = 10 the algorithm finds enough freedom 
to produce a more compact partitioner and at the same time keep the highly contributing pixels 
inside the partitioner. The average CVX runtime for each instance of the experiments in this section 
was less than 2 minutes. 


5.3 Challenging OCR Scenarios 

As the proposed technique allows us to identify the principal shape elements through a segmentation 
task, it would fit very well into an optical character recognition (OCR) application, were the primary 
objective is the identification of letters appeared in an image. The proposed algorithm would be 
capable of handling challenging OCR problems were the constituting letters are overlapping or 
misaligned, and the underlying image is cluttered. 

As the first basic experiment in this category, we consider the identification of the letters inside 
an image of size 63 x 160 pixels (shown in Figure 16) which represents the word “ SampLE”. The 
letters in the image undergo different case, size, rotation and level of overlap. To build up the 
character dictionary, yet maintaining a reasonable-sized dictionary, we use characters of a similar 
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Figure 14: The segmentation results for the test image in Figure 13 'b); results of different settings 
presented on the left column, the corresponding Ca.(x) depicted in the middle column and the resulting 
ol vector plotted at the right column, (a) r = 4; (b) r = 6; (c) r = 7; (d) r - 8; (e) r = 10; (f) Using the 


technique proposed in [3j 
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Figure 15: The segmentation results for the test image in Figure 13 'd); results of different settings 
presented on the left column, the corresponding Ccx(x) depicted in the middle column and the resulting ql 
vector plotted at the right column, (a) r = 3; (b) r = 5; (c) r - 6; (d) r = 7; (e) r = 8; (f) r = 10 
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font, but in various uppercase/lowercase formats. The character shapes are placed throughout the 
image in different size and orientations. With this setup, we build up a dictionary of n s = 2056 
elements. 

The segmentation results are shown for various values of r in Figure [l6j followed by the plots 
of the corresponding ol vector. A simple index map relating each index in a to the corresponding 
alphabetical representation, indicates the active letters in each segmentation problem. Again, suc¬ 
cessive increments of r from 1 to 6, allow us to identify the constituting letters one after the other. 
It is worth highlighting the transition of the results from r = 3 to r = 4, and how a slightly different 
letter “m” is identified to better match the overall shape. 

In Figure pT^g), we have again compared the results against the method in 13], where the letters 
with the largest weights are highlighted. In fact, the approach taken in [3] highly relies on an 
accurate alignment between the dictionary elements and the objects in the image. When this is not 
the case, a multi-stage reconstruction is proposed, where after each reconstruction, the dictionary 
elements with smaller weights are eliminated and are replaced with instances of the large-weighted 
letters at a finer resolution. This process is continued until convergence to a local minimizer is 
achieved. Clearly, using the proposed convex scheme allows us to perform the task more reliably 
and in a single step. 

In Figure [T7| we have presented the OCR results for a problem of similar setup, where the test 
image is noisier and the overlap among the characters is more significant. The discrepancy (relative 
to the ideal binary image) resulted by the clutter and the Gaussian noise in the image is more than 
90%. Similar to the previous noisy cases, and u ex are selected to be the 15% and 85% quantiles 
of the image histogram. Again successive increments of r from 1 to 6, allow us to identify the 
characters which build up the optimal partitioner. 

In this case that we have a tighter spacing among the letters, for r = 4 (Figure |T7|(d)) we 
observe that Sparse-CSC identifies a letter “d” instead of identifying one of the letters “a”, “p” or 
“L”. This is clearly because such selection causes more overlap of the resulting partitioner with the 
dark region in the image and hence produces a lower segmentation cost. For larger values of r the 
remaining characters are successively identified. Ultimately, Figure pT^g) shows the segmentation 
result without the G-constraint (he., r = oo), where all the dark regions are captured, however no 
inference about the principal shapes can be made. The average CVX runtime for each instance of 
the aforementioned experiments was approximately 8 minutes. 

As an additional OCR experiment, we consider a cluttered version of the Georgia Tech logo, 
where this time the letters in the dictionary take a different font than the version appeared in the 
image. The dictionary consists of 1068 letters of various size, rotation and placement. Still the 
striking performance of the method for successive values of r is presented in Figure [l8j We would 
like to note that the dictionary consists of a letter “C” very close to where the letter “G” has appeared 
and yet the algorithm identifies the true letter. We have also presented the segmentation results 
without an G-constraint (he., r = oo) in Figure pl^e). One can clearly observe the regularizing effect 
of the G constraint in the course of reconstruction. The average CVX runtime for each instance of 
this experiments was approximately 1 minute. 

5.4 Applications in Dense Packing 

Packing problems are a class of problems concerned with the arrangement of given objects inside 
larger containers |17 . Specifically, a relevant application that might be immediately linked to 
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Figure 16: An OCR problem addressed by segmentation: the red contour in the left column pictures 
shows a level-set of £cx(x) at some value between zero and one. The right column corresponds to the 
reconstructed weights and an indication of the letter each index corresponds to; (a) r = 1; (b) r = 2; (c) 
r = 3; (d) r = 4; (e) r = 5; (f) r = 6; (g) Using the tecggiique proposed in 3 







































































Figure 17: A more challenging OCR problem addressed by segmentation: the red contour in the left col¬ 
umn pictures shows a level-set of C (X (x) at some value between zero and one. The right column corresponds 
to the reconstructed weights and an indication of the letter each index corresponds to; (a) r = 1 ; (b) r = 2 ; 
(c) r = 3; (d) r = 4; (e) r = 5; (f) r = 6 ; (g) r = oo 49 















































































Figure 18: Another OCR problem where the image and the dictionary use different fonts: the red contour 
in the left column pictures shows a level-set of Ca.(x) at some value between zero and one. The middle col¬ 
umn depicts Col{x) and the right column corresponds to the reconstructed weights along with an indication 
of the letter each index corresponds to; (a) r = 1; (b\-jr = 2; (c) r = 4; (d) r = 5; (e) r = oo 
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Figure 19: (a) Twelve pieces of a puzzle to be put together; (b) Building up the shape dictionary by 
placing only one piece at the bottom left corner and placing rotated versions of the remaining pieces at 
the anchor points indicated by dots; (c) Each puzzle piece is rotated at a multiple of 90 degrees around an 
anchor point 


the material presented in this paper is densely packing a container with the minimum number of 
(non-overlapping) objects in M d , d = 1,2, *-. 

The packing problems are generally hard combinatorial problems. For instance, the two- 
dimensional rectangular packing problem, where both the container and the objects are rectangular 
geometries is shown to be NP-complete [l9j|. In higher dimensions and the case of irregular objects, 
the problem clearly becomes more complex and yet regarded as an NP-complete problem 121 j. 

Depending on the problem size and the quality of the shape dictionary, the approach presented 
in this paper may be employed to address instances of the packing problem, where the objects and 
container have irregular geometries. 

As a proof of concept, we consider the basic example of solving a 12-piece jigsaw puzzle, where 
the elements of the puzzle are shown in Figure [T9{a). Considering the puzzle pad to be S c T, the 
goal would be to tile E with the puzzle elements. For a shape dictionary consisting of the puzzle 
elements, this problem may be cast as performing the proposed segmentation task over an image 
defined in D, with unit pixel values inside E and zero outside. 

To avoid a large shape dictionary, we consider twelve equispaced anchor points within E, centered 
at which a puzzle piece may be placed. Clearly, for this example the puzzle pad is symmetric and 
there might be multiple ways of tiling E. For instance if a combination of the elements tiles E, 
a 180-degrees rotated version of this combination does the same job, and we may expect a non¬ 
uniqueness issue in the Cardinal-SC problem. This phenomenon is prevented by the way the shape 
dictionary is generated, as follows. 

As the first dictionary element, we place the shape corresponding to a true placement at the 
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Figure 20: Solving a puzzle by shape composition. The values of the resulting Ccx(x) shown as a colormap, 
and the corresponding a plotted below each reconstruction; (a) r = 12; (b) r = 11; (c) r = 10 


bottom-left corner anchor. This would be the sole element of the dictionary that is placed in this 
location, assuring that the Cardinal-SC problem yields a unique solution. For the remaining eleven 
anchor points, at each location we consider four rectangular rotations of the pieces of the puzzle 
excluding the bottom-left piece. This process is illustrated in Figure p^b)-(c), which yields a 
dictionary of size 1 + 11x11x4 = 485 elements. 

Figure [20ja) shows the segmentation result for r = 12, which indicates a successful completion 
of the puzzle. Panels (b) and (c) show the results for r = 11 and r = 10. It can be observed that in 
the latter case, the convex program is not able to maintain the sparsity level to the values indicated 
by r (i.e., 11 and 10), and to minimize the convex cost, E is covered by bringing into play more 
number of dictionary elements. 

One way to interpret the algorithm’s success in values of r below the true inclusion’s value, is the 
toleration against the possible inaccuracies in the given setup. In other words, the reconstruction 
failure for r < 12 in the preceding experiment, somehow reports a low toleration against the non¬ 
ideal setup (phenomena such as the LOC violation and inaccurate shape element alignments). 

As a second example, we consider a 9-piece puzzle with a similar setup as the aforementioned 
example, using 1 + 8x8x 4 = 257 shape elements. Figure [2T| shows the successful reconstruction 
results for r = 9, 8, 7. The experiment justifies the higher toleration of the proposed scheme against 
the possible sources of error when the true object is comprised of fewer components. 

6 Future Extensions 

Our emphasis on the binary image segmentation problem is to avoid unnecessary complication in 
the presentation and analysis. Several ideas developed in the paper are extendible to the multi¬ 
phase segmentation problem. Moreover, the idea of composing prototype shapes to represent more 
complex geometries extends beyond the context of image segmentation and can be employed in 
other shape-based imaging disciplines. 

In this paper we specifically focused on the composition rule (|3|, which was earlier presented 
in 131 and provides a flexible, yet simple model to combine the shape elements. A key component of 
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Figure 21: An attempt on a smaller puzzle with successful reconstructions for consecutive values of r; (a) 
r = 9; (b) r = 8; (c) r = 7 


our analysis is Theorem 4.2 which particularly focuses on the proposed composition rule. Depending 
on the application, we may consider other composition rules and inspired by the ideas developed 
in this paper derive new convex models. However, such extension requires stably relating the 
representations in the shape-domain and a-domain as performed in Sections |4.1| and |4.2| 

In Section |4.3| we showed the possibility of verifying the unique optimality of a given vector 
for the Sparse-CSC program. The optimality is cast as verifying the existence of the solution for 
a constrained system of equations. Although the conditions stated in Section |4.3| are general, to 
maintain brevity in Section p~4| we combined the results under the assumption that LOC holds for 
the target composition. These results can be extended to a more general case, where the LOC 
violation is sufficiently small. Such extension would provide us with quantitative results regarding 
the required contrast between the target shape and the background. 

While the basic convex formulation of the Sparse-CSC problem allows us to employ standard 
solvers, developing efficient optimization algorithms to numerically handle large-scale problems 
remains a future avenue of research. Specific interest would be on fast, memory-efficient and 
parallelizable techniques which allow breaking down the problem into smaller sub-problem (e.g., 
combining the composition results from different sections of the image, processed in parallel). Such 
algorithms may be equipped with updating rules of the inhomogeneity measures in the course of 
shape reconstruction. 


7 Proof of the Main Results 

The technical proofs associated with the presented results are detailed in this section. 

7.1 Proof of Proposition |2.1| 

For j e X® we define X® (x) = XSj • Clearly, 

{x:L®{x)>l}= (J S r 
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We set r]o = sup^^ L®(x) and for an arbitrary 77 > 770 > 1 we define 

£«0) = E VXSj- 
J&e 


In this case we have 


{x:l£(x)>r]}= U Sj. 

JtZe 


We now show L^x) = L®(x) - L®(x) meets the requirements of the problem. Clearly, 
L®(x)-L*(x) 


[ =L*(x)> 1 x € (U je i e Sj ) x (Uj<ex 0 Sj ) 

- 0 x 6 (U :/ €i e ) n (Ujei e <5j) 

= (^) < 0 X 6 (UjeX e <$j ) X (Uj e X ffi <Sj ) 

<0 X £ D X Uj 6 x ffi Sj 


For arbitrary sets A, and B cC we have 

C x (A x 5 ) = (£ \ A) u (B n A) u (C x A), 

and as a result 

DxE = Dx(( U Sj)x(U Sj)) 

jeX© jeX e 

= ((U <5i)x(U <5j))u((U Sj)n(U Sj))u(z?x \J Sj ). 

jeX e jeX® jeX® jeX e jeX® 


Combination of (70) and (71) yields 


£%{x)-L%{x) 


>1 X € S 
<0 x <e D \S ’ 


which completes the proof. | 


(70) 


(71) 


7.2 Proof of Proposition |3.7| 

We know 1^ and Xq are the only non-redundant index sets that ,x| ~ Therefore for any 
other non-redundant representation S' = %x' @ ,x' Q i we have S' % S or basically S' n (D \ S) ^ 0. By 
evaluating the SC cost for S' under the LOC we get 

I (n-m(^) — n e;E (x)) dx — I (n^ n (x) — n ea ,(x)) dx + I (rii n ( < x s ) — n ea ,(x)) dx 

' E'nS S'n(flxS) 

>/( n m(^) n-etc(^)) dx + / ( n m(^) fleccC^)) 

E'nE En(DxE') 

= (n in (x) - n f;:: , ; (.r)) dx, 

where the inequality is trivially true due to the sign of each term. Therefore, any S' $ S generates 
a cost strictly greater than the cost evaluated for S. Since S has a unique representation, {Z^, 1^ } 
is the strict minimizer of the SC cost. | 
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7.3 Proof of Proposition |3.8| 

We define 

( H 0 - {x : L a (x) < 0} 
j Hi = {x : 0 < L a (x) < 1} 

1 = {x : L a (x) > 1 } 

For the sake of convenience we use the following notations throughout the proof: 

~ (Ilin(z) — i^-ea;(‘^)) and A ecc (x) — ^II ecc (x) — Il^ n (x)^ . 


Clearly, 

A^ n (x) max (Hq,(x), 0) dx = J A in (x)L ot (x) dx = J A in (x)L ot (x) dx 

D EiuE 2 (EiuE 2 )n(D\E) 

where the last equality is thanks to the LOC. In a similar fashion 

J A ex (x) min (L a (x), l) dx = J A ex (x) min (L a (x), l) dx 

D E 

= J A ex (x)dx+ J A ex (x)L cx (x) dx. 


EnE 2 


En(EiuEo) 


Also 


J A ex (x)dx = J A ex (x) dx + J A ex (x)dx, 


EnE 2 


En(EiuEo) 


combining which with (74) yields 


J A ex (x)mm(L a (x),l) dx = J A ex (x) dx - J A ex (x) dx. 

E En(EiuEo) 


D 


From (73) and (|75| we arrive at 

J A in (x)m&x(L a (x),0) dx- J A ex {x) min (L a (x), l) da; 

D D 

= J A in (x)L a (x) dx + J A ex (x) dx - J A ex (x) dx 

(EiuE 2 )n(£K£) £n(£iu£ 0 ) S 

>- JA ex (x)dx. | 


7.4 Proof of Theorem 13.91 


We continue to use the notations (72). 


(72) 


(73) 


(74) 


(75) 
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We first show if (24) holds, a* minimizes (23). For this purpose we evaluate each component 


of the convex cost assuming that ([24]) holds: 

J' Ai n (x) max (L a * (x), 0) dx = J A in (x) max (x), 0) dx 

D E 

+ / A in (x) max (L a * (x), 0) dx 

D \E 

= 0 , 


(76) 


where the first term vanishes by the LOC and the second term becomes zero using (24). Moreover, 
JA ex (x)mm(L a *(x),l) dx = J A ea; (x)niin(A Q »(x),l) da; 


f A ex (x) min (L a * (x), l) da; 
ike 

J A ex (x) dx. 


(77) 


Comparing (76) and © with the result from Proposition |3.8| we can see that for cx* the convex 
cost ( |23| ) reaches its lower bound and therefore a* is a minimizer. 

We now show if cx* minimizes (23) then ( [24] ) should hold. From Proposition 2.1 we know there 
exists cx e W 18 such that 

Loc(x) >1 x € E 
La(x) <0 x e D \ E 

Knowing ( [78] ) holds, a similar argument as ( [76] ) and © results in 


D D 

From Proposition |3.8| we know if cx* is a minimizer, then 

fA in (x) max {L a » (x), 0) da: - f A ex (x) min (L a > (x),l)dx = -f A ex {x) dx. 

D D E 



(78) 

-J A ex (x) dx. 

(79) 

s 


-f A ex (x) dx. 

(80) 


This is due to the fact that A ex (x) dx is not only a lower bound and according to (79) it is 
also attainable. Thanks to the LOC we have 


and 


J A in (x) max (L a * (x), 0) dx = J A in (x) max (x), 0) dx 

D EKE 

J A ex (x) min (£ a * (x), l) dx = h ex (x) min ( L a * ( x ), l) dx. 


(81) 

(82) 


Using ([81]) and ( |82| ) in ( |80[ ) yields 

f A in (x)m ax (L a *(x),0) dx + f A ex (x)(l - min(T CK ^(x), l))dx = 0. (83) 


D \E 
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Since the integrants in 83 are non-negative and A i n (x) > 0 for x e D \ E, and A ex (x) > 0 for x € £, 
we must have 

min (L a * (x), l) = 1 x e int(E) 
max (x), O) = 0 x € int(D \ E) ’ 


which leads us to (24). 


7.5 Proof of Proposition |4.1| 

(a) The proof of this part is straightforward: 

*i =£ *2 =* e {1,2, •••,«} a.t. jf l] tjf 2) 

0jCii)(5j) = (e (i2) (S j )) C => s 0. 

J ' U 3 

(b) We prove this part for the case = 2 n - 1 for which we use induction. The argument 
certainly holds for n - 2. We now assume the argument holds for n - 1, i.e., 


2 n_1 -1 n-1 


U ^=U<5i- 

*=i j=i 


We add <S n to the collection and perform a DSD to obtain Q r shapelets. Based on (25) by using an 
appropriate indexing we can formulate the Q! shapelets for the worst case = 2 n - 1, as 


( fiiHSn 2 = 1,2,2 rx “ 1 - 1 

0' = ] ny/cl(5 t c )n5 n * = 2"- 1 

( n cl(<S£) i = 2 n_1 + 1, 2” - 1 


Therefore 


2n g^TL— 1 — 1 ^ -j^ 

U ft = ( U (fi<n5„))u( U (n i ncl(5“)))u(nd(5?)n5 n ) 

2=1 2=1 i=l j=l 

( 2 n_1 -l \ n -l 

U n (S n u cl(S£))) j u ( Q cl(5f) n S n ) 

= ( 2 J'cfi. n £>)) u ( fl cl(Sf) n S n ) 

2=1 V j =1 

2 n-1 -l n-1 

= ( U ni)u(nd(5f)n5 n ) 

2=1 V j = l 

. 72-1 \ / / n~l . \ 

= (U^)u cl((U5i) ) n <S n j 

j=i V i=i / 

^n(USj) 

J = 1 

72 

= U^ 

j=l 
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Which completes the inductive proof. When uq < 2 n - 1 the argument still holds as neglecting the 
empty shapelets does not affect the proof. 

(c) We prove the claim for an arbitrary j = jo. We exclude Sj 0 from the collection and perform 
a DSD on {Si,£ 2 ?“*? S n } \ {S/ 0 } to obtain Q r shapelets. We again consider the worst case that 
riQ' = 2 n_1 - 1. Using the result from part (b) we certainly have 


U^= U <5,. 

i =1 l<j<n 

3*3o 


Consider momentarily that jo = n. For i 
shapelets are either in the form of [ 0 lx ( n _ 
associated with the Q' shapelets. Based on this argument and the definition of a shapelet in (25), 
we have 


€ X n , the constructor vectors associated with the Q 
),1] T or \J' ,1] T , where J r is a constructor vector 


U Q-i = ( 2 l_f 1 -K n ^jo) u (‘5/,, n ( n cl(<sp)) 

izZj 0 i =1 1 <j<n 

3*30 

= (Sjo n ( 2 U _1 u (^io n d(( U <S,) C )] 

2=1 \ 1 <j<n ' J 

3*3o 

= U'o n( U 5,))ukncl(( U 

1 <j<n \ 1 <j<n / 

3*30 3*30 

= Sjo n D 
= Sjo 1 

which completes the proof. | 

7.6 Proof of Theorem 14.21 

Let E = (Uj€X 0 Sj) \ (U jex e Sj ). We start by proving part (a). 

Consider jo e Z®. To avoid long expressions we use the notations 

S+ = $3 an d S- = U Sj. 

3*30 

The non-redundancy of E requires that excluding Sj 0 from the composition should cause a change 
in the measure of E. In other words, since in general 

(5 + x5_)£((5 J0 u5 + )n5_), 
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we must have ((<S :/IJ u S+) \5-) \ (<5+ x <S_) $ 0, viz., 

3D s.t. int(D) t 0 and Cl = ((<Sj 0 u 5+) \ SJ) x (5+ \ <S_) 

= ((<Sj 0 v $_) u (<S+ x <S_)) x (<S+ x S-) 
= (5 io x5_)x(5 + x5_) 

= (<S io n5_ c )n(<S>5_) 

= (<S jo n 5f n <S^) u (S jo n <S! n <S_) 

= S jo n5!n 


Another way of interpreting this result is there exists a region Cl c <S, 0 that 


at (Cl) * 0 and Vj e (Z® x {j 0 }) u I® : 


ClnSj = 0. 


(84) 


Based on Proposition |4.1| we know the DSD process performs a partitioning over the shapes (ne¬ 
glecting the overlaps on the boundaries which are null sets). In other words, the outcome of set 
operations u, n and x amongst the Sj, j e I® ul e , can always be written as the union of some 
shapelets, up to a null set. Therefore (84) implies that there exists a shapelet Cl c Sjo that satisfies 


Vj e (I® u X e ) and j * j 0 ■ Cl n Sj ~ 0. 


(85) 


It is straightforward to see that if 


{CluJ^^DSD^Sj}^!,), 


the only shapelet Cl c Sj 0 that satisfies (85) is the one with a constructor vector that is zeros 
everywhere except the jo-th element. 

To prove part (b), we take a similar strategy. Consider this time jo e 2©. With a slight reuse of 
notation, this time we define 

S+ = (J Sj and S- = [J Sj. 

jzZ® j £ ^e 

3*3 o 

Excluding Sj 0 from the composition should cause a change in the measure of E. Since in general 


(5 + x(5_u5 j0 ))g(5 + s5_), 


the non-redundancy of E requires 

SCI s.t. int(fi) + 0 and Cl = ( S+ \ <S_) \ ( S+ \ (<S_ u Sj 0 )) 

= (5+ x S-) n (5+ x (S- u Sj 0 )) c 

= (<S+ x 5_) n (<S+ u u Sj 0 ) 

= (5+ x S-) n ((5+ x S-) c u<S JO ) 

= S jo nS+nS^ 

This result basically means that there exists a region Cl c (<S J0 n <S+) such that 

int(fi) * 0 and Vj g Z© \ {j 0 } : f2 n = 0. 

A similar argument as part (a) justifies the existence of some shapelet Cl c <S J0 n <S+ such that 

V j e X© and j * jo'- Cl n Sj ^ 0. | 
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7.7 Proof of Proposition |4.3| 

We consider a DSD as 

The proof to each part is presented as follows: 

(a) From Theorem |4.2[ b) we have 


Vj €2©, 3i elj, s.t. Qi oSj n ( (J 5j.) 

j € ^© 


and for j' 6 l e . j't j : 


^ <S jf ~ 0. 


As a result, the constraint (36) corresponding to the shapelet Qi simplifies to 


«i 


OLj < -|/C, 

Since c (Uj e x 0 <%)> we ^ ave l^©^l - 1- In other words, any solution of the convex program (|37| 
needs to satisfy 


o' i -IK, 


Mi 


<-l. 

(b) For each j g! 0 , a result of Theorem |4.2[ a) is the existence of a unique shapelet c Sj that 
does not overlap with any other shapes Sjf, j' e (X® \ {j}) uX®. Since for all j e X® we set at to 
one, we must have 0\ = 1. 

(c) For a given j e X®, by construction, the set of constraints ( [35] ) assure that for every i e lj , 
ft < 0 (this is another way of saying C^(x) < 0 over int(f^)). We only need to show that for a 
given j € X©, it is not possible to have ft < 0 for every i e lj . 

Suppose this is the case and for a j e X®, Vi € Xj, ft < 0. This means none of the constraints in 


(37), that OLj contributes in, are active (attain an equality). But if this is the case, we can increase 


OLj away from ft until one of the inequalities becomes active. In other words, concerning the linear 


program (37), we can find a point that produces a larger cost than at , which contradicts the 

I I - l q 

theorem’s supposition. 

(d) To facilitate the proof, we prescribe the index sets to be 


7 © = { 1 , 2 , • 


and X ft = 


+ 1,—,n© + n©}, 


where n®, n e e N. We form the corresponding bearing matrix B ^ € {0, l} n ^ x ( n © +n ©) 5 through which 
ft = B^ol^ . To prove the claim, it suffices to show that B ^ is full column rank. 

By applying suitable permutation on the rows of B ^ we can reshape it as 


n® n e 


PB^~ = 

7 ! O' 
B x I 


B 2 
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Here, P is the underlying permutation matrix, I represents the identity matrix and the blocks B\ 
and B 2 are some binary 0-1 matrices. The structure of the top n® x (n© + n e ) block of PB ^ 
is a direct result of Theorem |4.2[ a) that guarantees to have n® independent constructor vectors 
that are all zeros, except exactly one unit-valued element that occurs between the indices 1 and n®. 
The middle block structure [B i I] is thanks to Theorem |4.2[ b) that guarantees to have at least 
n e constructor vectors that are all zeros on the index set X®, except exactly one active index the 
location of which differs for every j e X® (Figure [6] and the example at the end of Section 4.1 can 
be helpful here). 

Demonstrating the aforementioned structure for PB ^ proves that it is a full column rank 
matrix, regardless of what the binary matrices B\ and B 2 are. | 


7.8 Proof of Theorem 14.51 

The claim is a corollary of the following lemma. 

Lemma 7.1 Suppose cx^ eW 1 is a maximizer of the linear program 

max l r a s.t. Aa < b (86) 


and T = {i : Ai^-af - bi}. 

(a) There exists a vector w > 0 such that 


(A t .) t w = 1. 


(87) 


(b) If rank(Ar, : ) = |T| = n and the essentially non-negative solution of (87) is strictly positive, then 


ex' 


is the unique maximizer of (86). 


(c) If |T| = n and cx^ is the unique maximizer of (86), then rank(Ar , : ) = n and the solution of (87) 
is strictly positive. 


Proof: Parts of the proof follow similar lines of argument as |23J, which are appropriately modified 
and included here to keep the paper self-contained. 

The dual linear program associated with (86) is 




min 

z 

S.t. 


b T z 

A t z = 1 • 
z>0 


Considering to be a minimizer of the dual problem, the optimality of af requires z^ c = 0, and 
subsequently (Ar, : ) T £p = 1. To establish part (a), it suffices to set w to z\. 

To prove the uniqueness of ol\ suppose there exists another maximizer a. ± af. Based on the 
linearity of the problem, any convex combination of the maximizers, such as (a^ + a)/2, is also a 
maximizer. More specifically, 1 T a) = 1 T (a) + a)/2 or 

l T (a f - a) = 0. (88) 


The feasibility of the maximizer also requires 


— Ay,-.oi) + — A^^ot < = A-p^.cx\ 
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or equivalently 

Ar, : (a^ - ot) > 0. (89) 

Suppose there exists j e T such that 

Aj^a^ - 6 l ) > 0 . 

Based on the lemma’s assumptions we have w = z\, > 0 and subsequently 

0 < Zp Ar, : (a^ - &) 

= A(oJ - a) 

= l T (a^ - a) 

= 0, 


which is not possible. As a result, Ar, : (a^ - a) = 0, that given the full rank property of Ar, : , is 
only possible when a = a^. This completes the proof for part (b). 

To prove the necessary condition (c), we will make use of the following theorem which is a 
variant of Theorem 1 proved in |23|: 


Theorem 7.2 A solution af of the linear program (86) is unique, if and only if for any 6 e W 1 
there exists a real positive number e such that oc t remains a solution of the perturbed linear program 


max (1 + eS ) T ol s.t. Aoc<b. 

(X 


Suppose y t is a solution to the dual program of (|90[), which is cast as 


mm 

y 

s.t. 


b T y 

A T y = l + e(5 
y>0 


(90) 


We first assume Ap, : is rank deficient and the solution to ( [86] ) is unique, and exhibit a contradiction. 
Based on this assumption rank(Ar, : ) < n and we can find a vector <5 q + 0 such that 


Ap,:(5o - 0. 

On the other hand, for any (e, 8) pair used in ( |90| ) we have 

eS = (A r , : ) T (y^-4)> 

which holds in view of y\ ,. = 0 and 


(91) 


{A T .) T y ] r = +V 
= l + e<5 

= (A r -) T z[ + e6. 


If we set 5 = <5 q, the uniqueness of the solution to J86|) warrants the existence of eo > 0 for which 


(91) holds. A pre-multiplication of both sides by <5 (l yields 


0 < e 0 <y<5 0 = 6%(A r ,) T (y{ - z\) = 0, 
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which is a contradiction. 

As the other possible case, assume that rank(Ar, : ) = n but the solution of (87) is not strictly 
positive. In this case we have Y - T' u T", where 

r' = {i : A^.oJ = bi,z\ > 0}, and r" = {*: Ai-cx ] = bi,z\ = 0}. 

Consider a vector £ e W 1 such that 

= 0, and Cf" = 1 • 

Since rank(Ar, : ) = n, the equation 

^■r,:^o = I 


has a non-zero solution S q. Again, appealing to (91) and taking into account that Y = T'ur" yields 


eS = (A r ',:) T (j/p, - Zp,) + (A T ^-) T (y ] T „ - z\„) 


*T’ V > 

\*7„t _„t 


= (ArvHi/^ - 4') + (■ A v „.) T y[„. 


(92) 


If the solution to (86) is unique, using Theorem 7.2 we can set S to £o and expect to have eo > 0 

— - ~ T 

for which (92) holds. Again a pre-multiplication of both sides by £ 0 yields a contradiction as 

0 < e 0 S 0 So = S 0 (Ap ): ) T (y{v - zp,) + £ 0 (^■r ,, ,:) T 2/p^ 

= 0-l T yp 
< 0 . 


In other words, if rank(Ap j: ) < n or the solution of (87) is not strictly positive, the solution of (86) 
cannot be unique. | 


7.9 Proof of Proposition 4.6 


We first show the following lemma holds. 

Lemma 7.3 Under the conditions stated in Proposition \4^\ for any at e M n % if ||a||i < r, then 

C r Ct < T. 


Proof: Suppose the claim is not true and there exists oc such that 

Nil ^ T 

and 

T 

C OL> T. 


(93) 

(94) 


Since cr e {1, —l}l r l we have 


E < E \aj\, 

j* r i«r 
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using which in (94) gives 


Consequently, 


£ l«jl + £ cj&j > t. 

jeF je F c 


S \^o \ < S c 3®3 - I S C i^il - ll c r c ||oo ^ 


( 95 ) 


(96) 


jer c 








where the first inequality is a direct result of combining (95) and (93), and the last relation is 
basically the Holder’s inequality. Ultimately, we must have 

(1 - |cr<=|oo) £ \6tj\ < 0 
je r= 

which is only possible when ||crc||oo > 1. This would be a contradiction with the assumptions made 
about c. | 


We now proceed by showing that a* is the unique minimizer of (44). Suppose this is not true 
and there exists 6l± oc* such that G(ct) < G(ot*) (i.e., 6t e C) and ||a|| < r. By Lemma 7.3, we must 
have 


T 

C (X <T. 


(97) 


On the other hand, by definition of the tangent cone 

3e > 0 s.t. ct* + e5 e C. 
Setting a = cG + cS. we must have ct eC and 

c T ct = c T ol* + ec T S > r. 

Consider now the line segment connecting ct and ct represented by 

L = [toe + (1 - t)&L : t c [0,1]}. 


(98) 


By the convexity of C we have L c C. Also, the continuity of the line segment along with (97) and 
(98) require us to have a z e L such that c T z = r. The point z does not need to be the same as a*, 


because to have that, ct needs to lie on the line passing through ct* and ct. Since C has a nonempty 
interior, we can always choose ct somehow that this is not the case. In other words, z is another 


minimizer of (49) and this is a contradiction with condition (I). § 


7.10 Proof of Theorem 14.71 

As discussed at the beginning of Section |T3} using the variable change (3 = Bex makes the objective 


function in (49) separable and the corresponding convex program is cast as 


min G((3) 

(3 


s.t. K 




V 

ct 


r 


(99) 
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where 


K = 


-I 

0 



We start the proof by the following lemma, which provides a set of sufficient conditions for optimality 
and uniqueness, while treating the linear constraint in a rather general way. We will then discuss 
how these condition translate into the ones stated in the theorem. 


Lemma 7.4 Consider ex* e R ns and P* = Bex * e R nfl , which together are feasible for the convex 
program and all the entries of (5* lie outside the interval (0,1). If there exists a vector 

A € M n ^ +n * in the range of K T such that 


A^ € (-qe,pi-qe) ^r 0 
A t z (pi - qi,pt) I tTi 


and 


and for 


II 

^€Ti + 


\ A, = -<# 

i e r 0 - 

(100) 

i A^=0 

(■ e {w-n + 1) •••, tiq + ji s } 


{nn + 1,—, 

nn +n s } 

(101) 


the matrix K : p' is full column rank, then (3* is the unique minimizer of (99) (and accordingly, ol* 
is the unique minimizer of m- 

Proof: We start by defining an objective function Q(.) : M n ^ +ns R: 


Clearly, an equivalent formulation of the convex program (99) is 

S(ffl) K 


mm 

/ 3,a. 


( 102 ) 


Using basic subgradient calculus |6j, it can be verified that a vector z e M nf2+ns is a subgradient of 
G(.) at [(3 t , a T ] T (denoted asze dQ{[(3 T , a T ] T )) when zi = 0 for I g {n^ + 1 ,-,nn + n s } and 


zi*\-qi,Pi-qi] • ^ = o 
: Pi = i 


Z£=P£ 

and < Z£=pe~qi 

{ Z£ = -qe 


Pi > 1 
0</3*<l 
Pi < 0 


Since /3^ £ (0,1) for all £ g { 1 , - ,nn}, (100) implies that A g 9</([/3* , a* ] T ). 

To prove that [/3*^a* T ] is the unique minimizer of (102), we need to show that any other 
feasible vector [/3 , a T ] ^ [/3*^a* T ] generates a greater cost, i.e., 




We proceed by defining a difference vector 


h = 

'0 


'0*' 

* 


a 


a 
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From the basic properties of the subgradient we have [6] 


Mz e dQ{^ 





( 103 ) 


As (3} i (0,1) for all £ € {1 for the index set r' in (101) we can consider the complement 

set 

r ,c = r 0 ur 1 , 


and decompose A and z e dQ([(3* T , a* T ] T ) as A = TV (A) + 7V c (A) and z = 7 D v r {z) + Vr fc (z ), 
where TV(-) is the projection operator onto the index set A: 



[ p ^)]' = {o' ‘el a- 

TV (A), (|103[) can be rewritten as 

m T ^ T \ 

\ T )>g([FW T ] T ) + h T z 

= g([/3* T , a* T ] T ) + h T ( A - Vrrc(A) + Pp,= (z)) 
= g([(3* T , a* T ] T ) + h T Vv c {z) - h T V r ^( A), 


(104) 


where, to establish the last equality we used the fact that Kh - 0 and A is in the range of K T . 

Based on the sign of the entries in /i, we can partition r /C into disjoint index sets denoted as 
follows: 


r' c = (r 0 n{£:^>o}) u (r 0 n{*:/*<o}) 


i° x° 

u (ri n {£ : he > 0}) u (ri n {£ : he < 0}). 


xi 


Zl 


Since inequality (104) needs to hold for any z e dQ([f3* ,oc* ] T ), we can narrow our choice to a 
specific subgradient vector y such that y T , = Ap' and 


Pe-qe u Ji 

Ve = i ^ x - 

Pe 


(105) 


As every entry of y is set to an extreme possible value in V' , (105) and (100) trivially reveal that 

y £ - Ai > 0 
y £ - Ai < 0 £ 

Based on this observation, we set 

6 = min ^{y £ - : £ e 1® u l\} u {A^ - y £ : £ e X°_ u X*} j, 
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which, based on the properties stated for y and A, is a strictly positive quantity. Accordingly, 


h T (Vr fC (y) ~ 7V c (A)) = ^ hi(y £ -Ai) 

ieF' c 

= E y h e (A e -y e ) 

ieZ%Zl ieZ%Z± 


using which in (104) gives 


«( l M( a* HMl 


- E Mi 

ieT rc 

^ E 1^ 

rer' c 


0 


’ - A/ 


(106) 


Since 5 > 0, we always have G{[fl , cL r V)>G{[p*‘ ^a* T ] T ) except when hr' c = 0 that an equality 
can happen. The possibility of having h ± 0 when hr rc = 0 is abolished, as K-^ is full rank and 
the linear equation Kh = 0 subject to hr' c = 0 has the unique solution h = 0 . In other words, the 
equality in (106) only happens when 


T 


’p*' 

a 


(X* 


^ 1 

Since for any pair of vectors (3 and cx related by f3 = Bex , we have £([/3 T , a T ] ) = g(/ 3 )=G(a), 
the preceding line of argument reveals that (3* is the unique minimizer of ( |99| and accordingly cx* 
is the unique minimizer of (49). | 


We can use the result of Lemma |L4| to complete the theorem’s proof. To show the full column 
rank property of K-^ we have 

-■Civurv B 


K 


L o- 

0 


By applying basic rank preserving operations we can reform K : ^ r as 


K 


,r' 


-I 

0 

0 


0 

? r 0 uri,: 


where the identity block has a width |To- u Ti+|. Clearly, if [(.B roUri} .) T , c] has a full row rank, 
K-^y' will be full column rank. 

Further, suppose there exist r] and r\ c that satisfy ([57]), (J58|. We introduce a vector f), the 
entries of which are set to be 

f Qi £ e IV 

Vi = \ ~Vi . (107) 

[ Vm(i) ^TouTi 


We show the vector K T [f) T ,? 7 c ] 
plete the theorem’s proof. 


satisfies conditions (100) of Lemma 


7.4 


Showing this will com- 
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Clearly, 


Jlc_ B T C _T ] c _ B T f] + T] c C_ ' 

By the construction (107), it is straightforward to verify that -f) e satisfies the conditions specified 
in (100) for t = 1, 2, no, and we only need to show that B T f] + r] c c = 0. For this purpose we have 

(^rhurv) ^?riur 0 + (-Br 1 + ur 0 -,:) f)r 1 +u r 0 - 

( T-> \ T _ . ~ T U) 


However, every element 


r] c c + B t t) = T] c C- 

= rj c c + (B ri ur 0 -.) T r] + E fi e J W 

= e + y feri+ur °" 

^r 1+ ur 0 - 

of the last expression vanishes since 

( E E 

£€r 1+ ur 0 - '3 &r 1+ ur 0 - 

E ve 

£e(r 1+ uF 0 -)rZj 


7 ? 


= E fie 

leZjC r 1+ 


E fie 

ieZj nr 0 - 


= - E Pe+ E to 

ieZjr\T 1+ leZjC\T 0 - 


= -e 


3- 


7.11 Proof of Proposition |4.8| 


From (39), it is straightforward to verify that 

r^urfv 


>!iV 




(108) 


where P e {0, i} n © xn © is a permutation matrix. The appearance of the permutation block is thanks 
to to Theorem |4.2[a). 


The invertibility of A^ for a basic composition warrants the invertibility of the matrix in (108), 
and hence the existence of a unique solution for (62). Accordingly, (62) may be cast as 


A* in * = 1 

v. r o _ 

P T w r n + Bw r n = -1 

1 i 1 n 


(109) 


where B = B 


( 2 , 1 )' 


The first equation in (109) is identical to (42), the strictly positive solution of which is a char¬ 
acteristic of basic compositions. Since A^ is a binary matrix and w r m is a strictly positive vector, 


its entries cannot exceed 1 and we must have 0 < w r ^ < 1. 


and 0 < w r ^ < 1, we immediately come 
validate the°claimed bounds on 


The solution to the second equation in (109) is simply w r ^ = -1 -PBw r ^. As PB e {0,1} 


n©xn e 


o-4he conclusion 4hat 0 < PBv8 v ^ < n©l. This would 
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7.12 Proof of Lemma 14.91 

We show the possibility of finding a point a in the interior of C, such that c T 6l > c T a* = ||a*||i. 

Consider ol' e M ns , where a^ ulQ = a^ ulQ and a -[x e uz e ) c = _e L The strictly positive scalar 
e « 1 is taken to be sufficiently small so that supp + (C ot '(x)) = E. 

Further, consider ol" = kct*, where k > 1. Thanks to the LOC, ol" is a minimizer of G(.) as it 
satisfies ( [24] ) . We set a = ol' + ol" . For a similar reason, a is a minimizer of G(.), which means 
G(a) = G(a*), or simply a e C. 

Finally, 

c T &- ||a*||i = &||a*||i -e Y c h 

je(Z®uZ e ) c 

where the right-hand side expression is capable of becoming strictly positive by choosing k suffi¬ 
ciently large. | 


7.13 Proof of Theorem 14.101 


Since LOC holds for cl(^x 9i z e ), using Theorem 4.7, we need to verify the possibility of finding a 
vector rj e Rl r ° uri l and a scalar tjc such that 


B' 

B" 


V + VcC = 0, 


( 110 ) 


where c I@ = 1, c Ie = 
of rj need to satisfy 


-1 and ||c(z ffiU z e )c ||oo < 1. Taking LOC into account, to satisfy (58), the entries 


-Pi=qi-Pi<'n i <q i = 0 i e To 
0 = -Pi < Vi < qi - Pi = qi i e Ti 


The cost function in (47) can be scaled by multiplying a constant, 
Therefore, the entries of v only need to satisfy 


without changing the problem. 


I Vi < 0 i e T 0 

1 Vi >0 i e r. 


( 111 ) 


We will show that when r/ c < 0 is chosen in a way that \t] c \ is sufficiently large, a vector with the 
following entries would satisfy the aforementioned requirements: 


1 i = 


ieJi, feryrf 

-1 ieT 


( 112 ) 


Verifying that (112) is in agreement with (111) is straightforward. To demonstrate that (110) holds, 
for j e X® we have 


Bj,. 1 ! X! l{ni<zSj}Wi X/ XI ^{QicSj} \ st \ w £ XI 

ieFouFi 2€j7jg i&T 


(113) 
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The second term on the right hand side of (113) is simply zero. 

we S et 


Using the fact that l{n iC Sj} = 


B j,.V Vc E I a \ 1 {nfaS j } We E tfiicnf} 
&r*ui? 

= r lc E 1 {nf.s j } w e 


ie rjurf- 

= -Vc- 

A similar line of argument shows that B r - 77 = r\ c for j el Q . We are only left to show the possibility 
of finding Cj e [- 1 , 1 ] such that 

Bj-nv-ne^ + VcCj = 0, j = n® + 7l e + 1, n s . 


By setting cj = -(%) l B" yq, we have 


j-n©-n e , 

1 


1 


\Vc\ ' i € T 

1 

\Vc\ ' i€T 

1 

\Vc\ l i 


Y + 7 ? c Y Tq~\ W L Y 

ter^-urf ' j£ ' 


Y + Vc XI lf£,j w l\ 


ieF^urf 


Yhtoi^Sj }M Y 


ieT 


p^i i 


When I ^Er^ur 51 < h we can take \q c \ to be sufficiently large so that \cj\ < 1 for all j e 

{n® +n e + 1 , —,n a }- | 


Appendix 

The following table provides a quick reference to some important notations: 
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Notation 


Description 


Notation 


Description 


D 

u(x) 

n in/ex (**0 
^ini Uex 

n s 


%©,X e 

Xs 

^Oi 

Oi , OLj 

supp + (.) 
int(.), cl(.) 
$ 

J 

©m 


imaging domain 
image pixel value at x 
inhomogeneity measures 
Chan-Vese mean pixels values 
number of dictionary elements 
a shape in the dictionary 
shape composition, see ^ 
index sets, see ^ 
set cardinality / magnitude 
see (7) 
see (7) 

shape coefficients, see |7 > 
positive support, see (11) 


interior and closure of a set 
see Section 


see (25) 


see (26) 


3.1 


n 

nn 

Zj 

B, B^ 

e(cc) 

P, Pi 

A(.) 

G(a) 

G{P) 

Pi, Qi 

To-, r 1+ 
r 0 , Ti 

e 

7e,j 

Ci 


a shapelet / cell 
number of shapelets 
see (27) 


dictionary / bearing matrices 
see (29) 


index sets, see (34) 


shapelet coefficients, see (|38) 
linkage operation 


discriminant matrix, see (41) 


convex cost, see (43) 


convex cost, see (47) 


see (48) 


see (53) 


see (54) 


LOC violation, see (56) 


see (65) 


shape coherence, see (68) 
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